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INTRODUCTION 


The  Enlisted  Personnel  Allocation  System  (EPAS)  is  a  classification  optimization  model 
that  is  designed  to  improve  the  efficiency  of  the  matching  process  that  links  recruits  to  specific 
job  training.  The  model  was  developed  to  work  within  the  existing  Army  training  reservation 
system,  known  as  REQUEST.  In  lieu  of  a  live  field  test  of  an  EPAS-enhanced  REQUEST 
system,  we  have  developed  a  simulation  field  test  to  estimate  the  classification  gains  of  the 
EPAS  enhancement. 

To  ensure  that  the  EPAS  Field  Test  Simulation  provides  a  realistic  and  unbiased 
evaluation  of  the  optimization  potential  of  EPAS,  a  model  simulating  Army  applicants’  job 
choice  decisions  is  needed.  This  report  summarizes  our  development  and  evaluation  of  an 
empirically-grounded  Job  Choice  Model  (JCM),  which  relates  applicants’  aptitude  scores, 
demographic  characteristics,  and  job  opportunity  attributes  (including  monetary  incentives)  to 
their  actual  choices.  As  with  real-world  applicant  decisions,  it  will  be  possible  under  the  JCM 
for  a  given  applicant  to  decide  not  to  join  the  Army  (not  access).  Similarly,  if  the  applicant 
elects  to  join  the  Army  (access),  the  JCM  can  simulate  the  applicant’s  choice  of  one  of  the  many 
MOS-reception-station  date  (job)  opportunities  from  their  job  list.'  By  sequentially  modeling 
actual  applicants’  choice  behavior,  the  JCM  provides  a  realistic  approximation  of  applicants’ 
decision-making  processes  for  simulation  purposes.  Evaluation  of  the  JCM  demonstrates  that 
the  model  effectively  simulates  applicants’  job  choice  decisions. 

This  report  is  organized  as  follows.  First,  we  summarize  the  JCM  and  our  approach  for 
mathematically  estimating  major  components  of  the  model,  particularly  applicants’  preferences 
(or  utilities)  for  different  job  opportunities.  Second,  the  procedure  employed  for  estimating  the 
JCM  is  described  and  results  evaluating  the  model’s  accuracy  are  presented.  Third,  and  finally, 
the  steps  required  to  simulate  applicants’  job  choices  for  purposes  of  implementing  the  JCM  in 
the  EPAS  Simulation  are  documented. 

MODELING  ARMY  APPLICANTS’  JOB  CHOICES 

The  main  goal  of  the  JCM  is  to  statistically  model  applicant  and  job  choice  charaeteristics 
for  purposes  of  simulating  applicant  job  choice  decisions  in  the  EPAS  Field  Test.  Conceptually, 
the  JCM  relates  attributes  of  alternative  job  opportunities  and  characteristics  of  applicants  to 
actual  choices.  Figure  1  summarizes  the  JCM  and  the  attributes  included  in  the  model. 

As  evident  from  Figure  1,  the  JCM  posits  that  Army  applicants’  job-choice  deeisions  are 
a  function  of  their  preferences  or  utilities  associated  with  the  different  job  opportunities 
presented.  These  preferences  are  related  to:  (1)  characteristics  of  the  applicant  (i.e.,  gender, 
education  level,  eognitive  aptitude,  etc.);  (2)  attributes  of  the  available  job  opportunities  (i.e., 
monetary  incentives,  rank  order,  etc.);  and  (3)  the  guidance  counselor  processing  the  applicant. 


'  For  simulation  purposes,  it  is  possible  for  individuals  in  the  applicant  data  who  actually  did  not  join  the  Army  to 
access  and  be  assigned  an  MOS.  Conversely,  it  is  also  possible  for  individuals  who  actually  joined  the  Army  to  not 
access  during  simulation  runs.  This  is  because  the  simulation  models  a  random  component  of  applicants’  job  choice 
decisions,  which  across  multiple  decision  events  would  function  to  produce  different  choice  decisions.  Note  that 
doing  so  increases  rather  than  decreases  the  accuracy  of  the  JCM  for  modeling  real-world  applicant  choice 
decisions.  More  importantly,  it  ensures  an  accurate,  unbiased  evaluation  of  the  optimization  potential  of  EPAS. 
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Consistent  with  the  aetual  decision-making  process,  the  JCM  produces  a  model  of  applicants’ 
choices  sequentially,  starting  with  their  decision  to  join  (or  not  join)  the  Army  followed  by  their 
choice  of  specific  job  opportunity  from  the  list  of  those  presented  at  time  of  enlistment. 

While  data  on  applicant  and  job  opportunity  attributes  and  applicants’  actual  job  choices 
were  available,  applicants’  preferences  or  utilities  are  latent  (or  unobserved)  variables.  To  model 
these  preferences,  we  applied  discrete  choice  modeling  and  random  utility  theory.  These 
modeling  approaches  have  been  widely  used  in  econometrics  to  model  consumer  choice  behavior 
(Greene,  1990)  and,  of  particular  relevance,  in  applied  psychology  to  model  Army  enlistment 
and  reenlistment  behavior  (e.g.,  Asch  &  Karoly,  1993;  Hogan,  Espinosa,  Mackin,  &.  Greenston, 
2002). 


The  rest  of  this  first  part  of  the  report  is  organized  as  follows.  First,  we  present  a  general 
overview  of  the  discrete  choice  and  random  utility  framework  underlying  the  JCM.  Second,  we 
develop  this  general  framework  to  explain  how  we  modeled  the  utilities  applicants  associated 
with  different  MOS  training  opportunities.  Third,  we  present  the  full  form  of  the  JCM  for 
predicting  applicants’  job  choices  based  on  these  utilities. 
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Overview  of  the  Modeling  Framework 

Our  main  motivation  in  developing  the  JCM  was  to  construct  a  mathematical  model  that 
closely  approximated  the  actual,  real-world  job  choice  process  of  Army  applicants. 

Operationally,  an  applicant  typically  goes  through  a  round  of  preliminary  processing  at  the 
MBPS,  then  sits  down  with  an  Army  guidance  counselor  to  determine  his/her  MOS  assignment. 
The  counselor  presents  the  applicant  with  a  number  of  MOS  training  opportunities  for  which 
he/she  is  eligible  based  on  test  scores,  demographics,  and  other  criteria  (e.g.,  physical  attributes, 
driver’s  license,  etc.).  From  these  opportunities,  the  applicant  makes  a  selection.  Alternatively, 
the  applicant  may  elect  not  to  join  the  Army. 

To  model  this  job  choice  process,  we  employed  discrete  choice  modeling  (McFadden, 
1974;  Train,  1986).  This  modeling  approach  is  commonly  used  in  econometrics  for  modeling 
the  choice  behavior  of  an  individual  decision-maker,  who  is  assumed  to  be  acting  rationally.  In 
constructing  the  EPAS  Simulation  JCM,  and  consistent  with  other  applications  of  discrete  choice 
modeling,  we  treat  the  applicant  as  the  sole  decision-maker.  It  should  be  noted,  however,  that  we 
recognize  the  important  role  that  the  guidance  counselor  plays  in  the  training  choices  of 
applicants  and  integrate  it  in  our  model  as  a  factor  defining  an  applicant’s  choice  situation. 

There  are  two  major  components  in  the  discrete  choice  modeling  framework.  The  first  is 
the  set  of  alternatives  from  which  the  decision-maker  chooses.  Technically,  an  applicant  at  the 
MBPS  is  deciding  on  a  training  choice  that  is  multidimensional,  as  characterized  by  the  MOS, 
reception  station  date,  location,  and  Term  of  Service  (TOS),  and  possibly  other  training 
reservation  variables.  Taken  together,  this  involves  a  very  large  number  of  discrete  alternatives 
that  is  difficult  if  not  impossible  to  model.  Given  the  specific  objective  of  our  analysis, 
predicting  applicants’  job  choices,  we  focused  mainly  on  the  MOS  dimension  of  the  training 
choice  for  the  purpose  of  defining  the  full  set  of  alternatives  under  consideration,  including  the 
option  of  not  joining  the  Army.  Other  dimensions  in  the  training  choice  decision,  such  as 
reception  station  date  and  TOS,  were  treated  in  a  secondary  manner.  Remaining  dimensions  of 
training  choice  were  not  considered  at  all.  To  make  the  alternative  dimensions  amenable  for 
model  estimation,  we  further  reduced  the  full  set  of  MOS  during  the  period  of  interest  (BY  2002) 
from  over  150  to  101  by  combining  comparable  MOS  with  very  small  reservations,  as  described 
later  in  this  report.  For  any  given  applicant,  typically  only  a  subset  of  these  MOS  alternatives 
will  be  available  in  the  job  list  presented  to  him/her  during  the  course  of  interacting  with  the 
guidance  counselor. 

The  second  major  component  in  the  discrete  choice  modeling  approach  is  the  rule  that 
governs  the  decision-maker’s  choice  process.  This  rule  is  based  on  the  assumption  that  the 
decision-maker  behaves  rationally.  In  our  problem,  we  assume  that  underlying  an  applicant’s 
training  choice  are  utilities  that  s/he  associates  with  different  MOS  training  opportunities.  Each 
utility  is  a  score  that  quantifies  the  value  of  an  MOS  alternative  to  an  applicant.  While  not 
meaningful  in  absolute  value,  utility  is  useful  for  studying  the  relative  attractiveness  of  MOS 
alternatives  to  the  applicant.  In  acting  rationally,  the  applicant  is  expected  to  choose  the  training 
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alternative  with  the  highest  utility.  Note  that  this  decision  rule  is  deterministic  given  the 
applicant’s  full  knowledge  of  utilities.^ 

Technieally,  the  utilities  used  by  the  applicant  to  evaluate  alternative  MOS  training 
opportunities  are  unobservable  to  the  analyst.  To  account  for  this  uncertainty,  utility  is 
represented  as  a  random  variable  in  discrete  choice  analysis.  We  will  use  to  denote  the 
utility  that  the  /th  applicant  attaches  to  the yth  MOS  alternative  (index  j  is  relative  to  the  full 
choice  set  of  MOS  opportunities).  While  the  exact  value  of  U^j  is  known  only  to  the  ith 

applicant,  it  is  reasonable  to  expect  this  value  to  be  related  to  the  attributes  of  the y'th  MOS 
alternative,  such  as  enlistment  incentives  and  bonuses.  Moreover,  this  value  is  also  likely  to 
depend  on  the  characteristics  of  the  applicant.  This  incomplete  information  on  the  applicant’s 
utility  is  reflected  by  writing  +  E^j ,  where  is  deterministic  utility  reflecting  partial 

information  and  Ey  is  a  random  variable  reflecting  uncertainty.  In  the  next  section,  we  will  fully 

specify  the  deterministic  utility  using  known  attributes  of  the  MOS  alternative,  the  applicant,  and 
the  guidance  counselor. 

Using  the  random  utility  U y  we  can  mathematically  present  the  general  form  of  a  model 

of  applicant  choice  behavior.  Suppose  that  the  /th  applicant  is  presented  with  m  MOS  training 
alternatives  identified  by  indices  1, 2,  ...,  m  {m<  101).  Since  we  do  not  have  complete 
information  on  the  applicant’s  utility,  we  can  only  give  a  probabilistic  statement  to  identify  the 
MOS  alternative  that  s/he  will  choose.  Specifically,  the  probability  that  the  applicant  chooses  the 
/rth  alternative  is  given  by: 

PXk)=P[U„=mz.x\jy\j  =  \,2,...,m\) 

=  >Uy  \j  =  \,2,...,k-\,k  +  ],...,m) 

=  P{v,-Vy>Ey-E,,\j  =  ],2,...,k-\,k  +  \,...,m) 

These  probability  statements  reflect  our  uncertainty  (as  analysts)  regarding  the  training 
choice  of  the  applicant  because  of  less  than  full  knowledge  about  the  applicant’s  utilities,  but 
they  do  not  alter  the  deterministic  nature  of  the  applicant’s  decision  rule.  The  first  line  above 
restates  the  assumption  (or  decision  rule)  that,  when  making  a  training  choice,  applicants  seek  to 
maximize  utility.  Therefore,  to  completely  define  the  choice  model,  we  need  to  fully  specify  the 

^  The  assumption  of  rationality  should  be  addressed  in  light  of  the  substantial  body  of  research  indicating  the  limits 
of  human  rationality  in  determining  choices  (e.g.,  Simon,  1955),  the  heuristics  that  are  used  to  evaluate  and  select 
decision  options  (e.g.,  Kahneman,  Slovic,  &  Tversky,  1982),  and  the  prominence  of  theories  that  relax  assumptions 
of  rationality  (e.g.,  Kahneman  &  Tversky,  1979;  Klein,  Orasanu,  Calderwood,  &  Zsambok,  1993).  Knowledge  of 
these  limits  would  suggest  ways  that  individual  choices  should  differ  from  the  assumptions  of  the  discrete  choice 
analysis  methodology,  in  which  choice  probabilities  are  used  to  estimate  utility  differences.  For  example,  we  would 
expect  utility  differences  for  opportunities  that  are  similar  in  many  dimensions  (e.g.,  similar  bonuses  and/or  MOS  in 
the  same  job  family)  to  be  more  salient  than  comparably  sized  differences  between  opportunities  that  differ  in  nearly 
all  dimensions  leading  to  more  extreme  choice  probabilities  for  the  similar  opportunities.  Such  deviations  of  choices 
from  the  assumptions  of  the  analysis  will  add  some  error  to  the  model  estimates.  Model  utility  estimates  will 
aggregate  choices  in  a  wide  range  of  opportunity  combinations,  and  consequently  provide  an  overall  average  value 
that  characterizes  the  cohort.  Thus,  we  anticipate  that  the  modeling  framework  will  be  relatively  robust  to 
deviations  of  applicant  choices  from  rationality  assumptions. 
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form  of  the  deterministic  component  and  distributional  assumptions  on  the  random  component 
utility.  Having  done  so,  we  can  expect,  as  stated  in  the  last  expression,  that  as  the  unexplained 
utilities  E^j  s  become  smaller  relative  to  the  explained  utilities  V^j  s,  the  closer  will  be  the 

correspondence  between  applicants’  predicted  training  choices  and  their  actual  (observed) 
training  choices  (i.e.,  probabilities  for  the  chosen  MOS  alternative  will  be  close  to  100  percent). 

In  sum,  the  main  goal  of  the  JCM  is  to  represent  the  applicant’s  decision  rule  as  a 
probabilistic  choice  model.  The  response  variable  in  this  modeling  problem  is  the  choice  of  an 
applicant  among  several  alternative  MOS,  while  the  explanatory  variables  are  attributes  of  the 
MOS  training  opportunities,  characteristics  of  the  applicant,  and  counselor  performance.  The 
model  “predicts”  choice  of  MOS  in  the  form  of  probabilities  attached  to  each  alternative  MOS  in 
the  job  list,  reflecting  the  relative  likelihood  of  each  being  chosen  by  the  applicant.  Given  its 
intended  application  in  the  EPAS  Field  Test  simulation,  to  predict  applicant  job  choice,  we 
primarily  focus  on  these  probabilities  and  not  on  the  underlying  utilities.  Focusing  on  the 
utilities  would  be  relevant  if  the  objective  of  the  analysis  was  to  inform  incentive  policies. 

Modeling  Applicant  Utility 

As  discussed  in  the  preceding  section,  and  as  is  common  with  most  prediction  problems, 
precise  information  regarding  variables  and  their  contribution  to  the  unknown  utility  of  a  specific 
applicant  for  a  given  MOS  is  impossible  to  obtain  with  absolute  certainty.  Most  important,  these 
variables  and  their  contribution  to  utilities  can  (and  will)  differ  from  one  applicant  to  another. 
While  we  can  pool  applicants  to  obtain  an  “average”  utility  that  applicants  who  share  some 
characteristic  “Z”  attach  to  an  MOS  alternative  with  an  attribute  “X”,  there  remains  a  residual 
utility  that  is  not  explained  by  this  average.  This  idea  is  analogous  to  traditional  regression 
analysis  and  is  a  motivation  in  partitioning  utility  into  deterministic  (or  systematic)  and  random 
utility,  U =  V^j  +  Eij ,  where  Vy  represents  average  utility  and  Ey  denotes  residual  utility,  which 
could  be  due  to  unobserved  MOS  attributes  and/or  applicant  “taste”  variations. 

As  a  first  step  in  constructing  the  JCM,  we  needed  to  model  applicant  utilities.  This 
required  the  following:  first,  fully  specifying  the  deterministie  utility  function  and  its  individual 
components,  those  variables  representing  applicant  characteristics  and  choice  attributes  expected 
to  explain  applicants’  training  choices;  and  second,  specifying  the  distributional  assumptions 
underlying  the  residual  (or  error)  utility  term.  In  the  following  sections  we  describe  how  we 
specified  each  of  these  components  and  operationalized  their  constituent  parts  within  the  EPAS 
Simulation  JCM. 

Specifying  the  Deterministic  Utility  Function 

We  specified  the  deterministic  utility  function  as  a  linear  deterministic  utility  Vy  using  a 

combination  of  transaction  variables  that  are  expected  to  reasonably  represent  “average” 
applicant  utility  and,  therefore,  choice  behavior.  These  variables  include  monetary  incentives 
offered  with  the  MOS,  demographics  and  aptitude  scores  of  applicants,  rank  order  of  the 
alternatives  in  the  applicant  job  list,  and  a  measure  of  counselor  performance.  The  last  two 
variables  are  essential  in  integrating  the  EPAS  optimization  in  the  JCM. 
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Let  the  MOS  alternatives  be  denoted  by  j  =  1,2, ...,101  and  the  non-accession  alternative 
by  j  =  999 .  We  partitioned  deterministic  utility  into  two  main  components  and  an  alternative- 
specific  constant  by  writing  as 


where 


Aj*V,(z)*V,(X,C) 
A  +V 

-^999  ^  /999 


(^) 


y  =  1,2,. ..,101 

j  =  999 


~  ^ Rnk.C RnkJ  ^ IsTEA^ IsTEAJ  ^SBd^ SBdJ  ^ABd^ 


ABdJ 


^HGd^ HGdJ  +  ^AA^AAJ 


^ij\^)~^sexMJ^Sex,i  ^ edNG J ^ edNG ,i  ^ edSJ^ edS ,i  ^ edGC J ^ edGC J 

^ AQA.J^ AQA,i  ^ Afqt .NA^ Afqt  ^RSj^RS.i 

The  first  component,  (X,  C) ,  is  deterministic  utility  that  depends  on  the  attributes  X  of  the 

MOS  alternative  (e.g.,  monetary  incentives)  and  a  measure  of  counselor  performance.  The 
counselor  performance  measure  (C)  is  incorporated  into  the  coefficient  ,  as  described 

below.  Note  that  this  component  was  dropped  from  the  utility  function  for  non-accessions  as  it 
describes  attributes  that  are  only  meaningful  to  MOS  alternatives.  The  second  component, 
kA^)  ,  is  deterministic  utility  that  depends  on  the  characteristics  Z  of  the  applicant  (e.g., 

gender).  The  full  set  of  MOS  alternative-specific  attributes  and  applicant  characteristics  used  in 
these  equations  are  summarized  in  Table  1  (below). 

Table  1.  List  of  Alternative-Specific  and  Applicant  Attributes  Included  in  JCM 


Variable 

Description 

Alternative-Specific  Attributes: 

^  RnkJ 

Relative  rank  of  the yth  MOS  alternative  in  the  job  list. 

^  LsTBAJ 

Expected  maximum  utility  of  applicant  for  the  EB/ACF/LRP  incentive 
package  available  for  the yth  MOS.  The  form  of  this  eomposite  utility  is 
given  below. 

^SBd, 

Seasonal  bonus  dollars  offered  with  the yth  MOS  alternative  (in  thousands). 

^  ABd.j 

Airborne  bonus  dollars  offered  with  the  jxh  MOS  alternative  (in  thousands). 

^  I  fGdJ 

High-Grad  bonus  dollars  offered  with  the yth  MOS  alternative  (in  thousands). 

^AAj 

Aptitude  area  score  of  the  applicant  for  the yth  MOS  alternative. 

Applicant  and  Counselor  Characteristics: 

7 

^  sexM  J 

Sex  indicator  variable  for  male  (l=Male,  0=Female) 

7 

^(idNG.i 

Indicator  variable  for  non-graduate  education  status 
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^edSJ 

Indicator  variable  for  senior  education  status 

^edGCJ 

Indicator  variable  for  education  status  beyond  high  school  graduate  (i.e.,  at 
least  some  college  semester  hours) 

7 

^AQA,i 

Indicator  variable  for  AFQT  Category  I-IIIA 

7 

^RS,i 

Indicator  variable  for  South  Region  geographic  location 

AFQT  percentile  score 

C, 

Measure  of  counselor  performance  based  on  the  60’”  percentile  of  the  ranks 
of  MOS  reservations  processed 

In  the  equations  above,  the  B-  and  G-weights  describe  the  relative  importance  of 
associated  MOS  alternative  attribute  or  applicant  characteristics  to  the  total  utility.  These 
weights  and  alternative  constant  Aj  are  parameters  to  be  estimated  from  the  transaction  data. 

Greater  detail  on  the  variables  representing  the  alternative  attributes  and  applicant  characteristics 
and  how  they  were  specified  in  the  JCM  are  summarized  in  the  following  sections. 

Rank  Order  Effect.  In  general,  MOS  alternatives  that  are  important  to  Army  enlistment 
goals  appear  at  the  top  of  applicants’  opportunity  (or  job)  lists.  Operationally,  this  can  be 
represented  by  the  variable  X j ,  where  the  rank  order  of  an  MOS  alternative  (within  a  job  list) 

is  expressed  as  a  percentage  relative  to  the  total  number  of  opportunities  in  the  list.  However, 
unlike  the  monetary  incentives,  this  attribute  (by  itself)  is  not  expected  to  directly  contribute  to 
applicants’  utility.  That  is,  an  applicant  is  not  expected  to  be  “attracted”  to  an  MOS  just  because 
of  its  rank  order  in  the  list.  Instead,  the  extent  to  which  an  applicant  selects  an  MOS  at  the  top  of 
the  job  list,  excluding  the  effects  of  monetary  incentives,  will  depend  on  the  guidance 
counselor’s  ability  to  “sell”  these  jobs.  A  higher  ability  counselor  should  be  able  to  sell  higher- 
ranked  MOS  compared  to  a  lower  ability  counselor,  when  facing  applicants  with  similar 
(observed  and  unobserved)  characteristics  and  job  lists  with  comparable  MOS  and  monetary 
benefits. 

Consistent  with  this,  we  expanded  the  operationalization  of  the  effect  of  rank  of  an  MOS 
on  applicant  utility  to  include  counselor  ability.  To  do  this,  we  computed  an  empirical  measure 
C,  to  index  the  ability  of  the  counselor  that  the  fth  applicant  faced  at  the  MBPS.  This 
performance  measure  was  based  on  the  60’*’  percentile  of  the  overall  ranks  of  MOS  in 
reservations  made  by  all  applicants  processed  by  the  counselor  during  the  period  covered  by  the 
estimation  sample.^  The  weight  (or  effect)  of  rank  order  attribute  X  j  for  the  ith  applicant  was 

then  reformulated  as  +  5^0, .  The  utility  term  corresponding  to  rank  order  then 

becomes 


^  The  overall  rank  used  to  compute  the  measure  C;  uses  the  rank  order  of  an  MOS  relative  to  all  MOS  that  were 
available  on  a  given  transaction  date,  and  not  the  rank  order  relative  to  the  job  list  of  an  applicant.  The  overall 
ranking  of  MOS  was  “estimated”  using  rank  ordering  information  from  job  lists  of  applicants  during  a  given 
transaction  date. 
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(1) 


^RnkjCi^ RnkJ  ~i^Rnk  ^ i)^ Rnk J 

~  ^Rnk^RnkJ  +  ^ci^i^Rnkj) 

From  the  second  line  expression,  the  rank  order  term  of  utility  may  also  be  viewed  as  a  main 
effect  plus  an  interaction  between  MOS  rank  order  in  the  job  list  and  counselor  performance.  It  is 
important  to  note  that  in  applying  the  JCM  to  simulate  applicant  choices,  an  assumption  is  that 
the  Army  priority  rank  ordering  and  the  combined  EPAS-Army  priority  rank  ordering  are  not 
distinguishable  to  a  counselor. 

The  contribution  of  the  rank  order  term  above  to  total  utility  of  the  applicant  represents 
“partial  effect”  as  in  typical  regression  analysis.  It  is  “partial”  in  the  sense  that  it  accounts  for  the 
applicant’s  utility  not  already  explained  by  monetary  incentives  and  other  factors  included  in  the 
utility  function.  This  note  is  important  since  monetary  incentives  and  rank  order  are  highly 
correlated  by  design.  A  utility  model  that  fails  to  properly  account  for  monetary  benefits  will 
overestimate  the  role  of  the  guidance  counselor  in  applicant  selection  of  “high  ranking”  MOS 
alternatives  (and  vice  versa).  That  is,  it  will  confound  counselor  ability  with  the  effects  of 
monetary  incentives  and,  therefore,  lead  to  biased  EPAS  Field  Test  results. 

Monetary  Incentives.  The  attributes  A,, ^  ,  and  X j  represent  Army 

incentive  policy.  The  first  attribute  is  a  composite  of  Enlistment  Bonus  (EB)  and  Army  College 
Fund  (ACF)  incentives,  which  are  tied  to  the  TOS.  Also  included  in  this  attribute  is  the  Loan 
Repayment  Program  (LRP)  package,  which  is  offered  in  place  of  ACF.  The  form  of  this 
composite  is  described  in  more  detail  below.  The  other  three  attributes  represent  distinct 
monetary  incentives.  As  with  EB/ACF,  the  dollar  values  in  these  incentives  differ  across  MOS, 
reflecting  MOS  importance  to  the  Army’s  enlistment  goals.  The  availability  and  dollar  amount 
of  incentives  can  also  differ  depending  on  the  applicant’s  qualifications  for  a  given  MOS.  For 
instance,  the  overall  value  of  the  EB/ACF  incentive  package  is  highest  for  1 IX,  reflecting  the 
importance  of  the  MOS  to  the  Army’s  mission.  This  incentive  package  is  only  available  to 
AFQT  I-IIIA  applicants.  The  purpose  of  the  Seasonal  Bonus  (SB)  incentive  is  to  encourage 
enlistment  into  and  fill  of  near  term  training  classes.  It  is  given  in  three  levels  depending  on  how 
close  the  start  date  of  a  training  opportunity  is  at  the  time  of  the  transaction.  For  a  given  SB 
incentive  level,  different  dollar  amounts  are  offered  to  AFQT  I-IIIA  and  IIIB  applicants. 

Similarly,  Hi-Grad  (HG)  incentive  dollars  are  available  to  applicants  with  some  college 
education  if  they  enlist  in  “incentivized  MOS”  (i.e.,  MOS  eligible  for  EB/ACF  incentives).  The 
HG  dollar  amount  also  differs  depending  on  whether  the  applicant  has  earned  at  least  30  or  60 
college  semester  hours.  Overall  the  net  effect  of  these  incentives  is  to  make  certain  MOS  more 
attractive  than  others  to  particular  types  of  applicants. 

As  mentioned  above,  the  variable  X -  was  computed  as  a  composite  of  cash  bonus 

and  ACF  dollars  computed  from  the  EB,  EB+ACF  combo,  and  ACF  incentive  packages 
available  to  an  AFQT  I-lIIA  applicant  signing  up  for  about  80  to  90  MOS.  By  design,  the  Army 
has  constructed  its  incentive  policy  such  that  the  availability  of  these  three  types  of  EB/ACF 
incentives  and  the  associated  dollar  amounts  depends  on  the  MOS  and  TOS.  For  high  priority 
MOS,  these  incentives  (EB  and  EB+ACF)  are  available  in  higher  dollar  amounts  even  for  short 
TOS  (2  or  3  years).  For  middle  priority  MOS,  EB  and  EB+ACF  incentives  are  offered  but  with 
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smaller  dollar  values  and  only  starting  with  at  least  4  years  TOS.  For  lower  priority  MOS,  bonus 
dollars  (either  from  EB  or  EB+ACF)  in  relatively  small  amounts  may  be  available  but  only  for 
longer  TOS  (5  or  6  years),  and  in  some  cases  only  ACF  is  available. 

We  treated  the  EB/ACF  component  of  utility  differently  from  the  others  for  several 
reasons.  First,  as  described  above,  different  types  of  EB/ACF  incentive  packages  are  available 
for  the  same  MOS.  This  situation  differs  from  the  other  incentives  whose  dollar  value  (and 
form)  stays  the  same  for  a  given  MOS.  Second,  unlike  the  other  incentives  that  are  independent 
of  TOS,  the  applicant’s  choice  from  the  EB/ACF  incentive  package  cannot  be  separated  from 
TOS-a  dimension  of  applicant’s  training  choice  that  is  not  important  to  the  current  problem. 
Third,  Army  incentive  policy  tends  to  treat  EB  and  ACF  incentives  interchangeably,  frequently 
combining  the  two  into  a  single  package,  such  that  the  incentives  represent  dependent  rather  than 
independent  effects.  For  these  reasons,  we  elected  to  integrate  the  EB/ACF  incentives  into  a 
single  variable  in  the  JCM. 

To  do  this,  we  formed  a  composite  to  represent  applicants’  expected  utility  from  the 
many  EB/ACF  incentives  and  TOS  possibilities  for  a  given  MOS.  We  also  included  the  LRP 
incentive  in  this  composite,  as  it  is  offered  in  place  of  ACF  for  some  applicants.  Separate 
composites  were  formed  for  AFQT  I-IIIA  and  IIIB  applicants  since  the  latter  are  not  eligible  for 
EB/ACF  incentives.  The  most  general  form  of  this  composite  is  given  by 

^imAj  =  logZ  log[exp(M,F;  )+  exp(M,F;, )+  exp(M,F; , )+  exp(M,F;^_,  )]| 

where  M,  is  a  positive  constant  that  depends  on  TOS  for  the  incentive.  The  terms  in  the  inner 
log  expression  correspond  to  the  different  types  of  EB/ACF  incentive,  respectively:  (1)  none,  (2) 
EB-only,  (3)  ACF-only,  (4)  EB+ACF  combo;  summation  is  over  number  of  years  (t)  of  TOS. 

For  a  specified  MOS,  only  terms  corresponding  to  EB/ACF  incentives  that  were  available  to  the 
applicant  are  included  inside  the  curly-braces. 

Taken  as  a  whole,  the  composite X j  aims  to  capture  the  applicant’s  expected  utility 

for  alternative  EB/ACF  incentives  and  TOS  for  the /th  MOS  alternative.  The  F  quantities  in  the 
composite  represent  utilities  associated  to  the  four  types  of  EB/ACF  incentives,  and  are  given  by: 

v;  =  A, +BJ, 

^E,l  ~  A  E,J  ^iJl 

yi,  =  A  *B,X,^,*B,X^,  *B,(hX^^) 

where 

X =  bonus  dollar  value  of  EB  incentive  for  a  TOS  of  t  years; 

Xp  ,  =  bonus  dollar  value  of  EB+ACF  incentive  for  a  TOS  of  t  years; 
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X ,  =  ACF  dollar  value  of  ACF  incentive  for  a  TOS  of  t  years: 

A\  ,l 

I  =  ACF  dollar  value  of  EB+ACF  incentive  for  a  TOS  of  t  years; 

Ij  ~  indicator  variable  representing  the  availability  of  the  LRP  incentive  for  the 

MOS; 

1^  =  indicator  variable  for  senior  education  status. 

The  LRP  incentive  was  represented  in  the  composite  using  an  indicator  variable  since 
only  maximum  loan  is  specified.  It  only  appeared  in  the  first  two  types  of  incentives,  as  it  cannot 
be  combined  with  ACF.  The  ACF  dollar  value  used  in  the  composite  is  less  than  the 
Montgomery  GI  Bill  amount  of  $23,400  dollars.'*  The  contribution  of  ACF  to  utility  includes  an 
interaction  involving  high  school  senior  education  status  of  the  applicant.  We  included  this 
interaction  because  seniors  can  be  expected  to  find  ACF  incentives  more  attractive  than  non-high 
school  graduates  and/or  high  school  graduates  with  some  college.  The  quantities  A  and  B  are 
parameters  to  be  estimated  from  the  transaction  data. 

Aptitude  Area.  The  variable  represented  by  X .  is  the  aptitude  area  (AA)  score  of  the 

applicant  corresponding  to  the yth  MOS  alternative.  It  is  the  only  alternative-specific  variable 
that  does  not  represent  Army  priority.  The  value  of  X ^  j  is  a  measure  of  the  applicant’s 

aptitude  for  the  type  of  job  that  characterizes  the  yth  MOS  (e.g.,  Clerical,  Mechanical,  etc.).  AA 
scores  play  important  but  meaningfully  different  roles  in  REQUEST  and  EPAS.  The  REQUEST 
system  uses  the  AA  score  as  a  key  variable  in  determining  the  eligibility  of  an  applicant  (e.g. 
only  MOS  whose  minimum  enlistment  standards  are  met  by  an  applicant  will  appear  in  his/her 
job  list).  The  EPAS  model  employs  the  AA  score  in  its  person-job-match  optimization,  which 
aims  to  identify  the  MOS  best  suited  for  the  applicant  subject  to  Army  enlistment  priority 
constraints.  Because  the  value  of  this  variable  is  believed  to  generally  reflect  the  vocational 
interests  of  the  applicant,  it  is  expected  to  contribute  to  their  utilities  for  MOS  alternatives. 

Applicant  Characteristics.  As  the  Army  intends,  the  monetary  incentives  discussed  above 
are  expected  to  make  some  MOS  more  attractive  than  others.  However,  their  overall  effect  on 
training  choices  is  not  likely  to  be  uniform  across  applicants.  For  example,  the  relatively  high 
EB  dollars  available  to  priority  MOS  (for  signing  up  for  a  TOS  of  six  years)  may  not  be 
appealing  to  a  high  school  senior  who  plans  to  go  to  college.  Likewise,  mechanical  jobs  are 
likely  to  be  more  attractive  to  male  than  female  applicants.  Because  of  these  differential  effects 
on  training  choices,  we  added  applicant  characteristics  to  the  utility  function  of  applicants. 

The  specific  applicant  characteristics  included  in  the  utility  are:  (1)  gender;  (2)  education 
status;  (3)  AFQT;  and  (4)  applicants’  geographic  location.  These  characteristics  were  selected 
for  two  reasons.  First,  these  are  known  to  impact  the  type  of  MOS  preferred  by  applicants. 
Second  and  more  importantly,  most  of  these  characteristics  are  relevant  to  EPAS  in  that  they 
define  the  supply  groups,  which  represent  applicants  in  the  EPAS  optimization  algorithm.  By 
including  these  characteristics  in  the  JCM,  one  can  obtain,  for  example,  percentages  of  applicants 


''  The  EB/ACF/TOS  composite  was  motivated  by  an  expanded  model  with  choice  dimension  (MOS,  TOS,  EB/AC 
incentive).  The  A  and  B  parameters  were  estimated  from  the  transaction  data  using  applicants  choice  of  MOS,  TOS, 
and  EB/ACF  incentive. 
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in  a  supply  group  that  will  prefer  alternative  MOS,  which  itself  could  be  directly  useful  in  the 
EPAS  optimization  routine. 

To  incorporate  applicant  characteristics  into  the  JCM,  indicator  variables  were  created  to 
represent  group  membership,  except  for  AFQT  percentile  score  which  was  treated  as  a 

continuous  variable  and  whose  values  reflected  applicants’  AFQT  percentile  scores.  Gender, 
represented  in  the  utility  by  ,  constituted  the  indicator  variable  for  male  applicants.  To 

further  capture  meaningful  differences  in  education  status,  the  three  categories  used  in  defining 
education  status  for  the  EPAS  supply  groups  were  expanded  to  four.  In  addition  to  non-graduates 
( '^edNG  i )  seniors  ( Z^^^ , ),  the  high  school  graduate  status  was  divided  into  two  separate 
categories:  (1)  one  for  those  who  earned  a  high  school  diploma  but  did  not  attend  college;  (2) 
and  one  for  those  who  attended  college.  The  latter  is  represented  by  the  indicator  variable  • 

AFQT  category  is  represented  in  the  utility  using  the  indicator  variable  Z^q^.  for  AFQT  I-IIIA 

applicants.  We  included  the  AFQT  I-IIIA  indicator  variable,  in  addition  to  the  percentile  score, 
because  separating  applicants  who  are  eligible  from  those  who  are  not  eligible  for  incentives 
provides  meaningful  information,  as  most  incentives  require  an  applicant  to  be  in  the  AFQT  I- 
IIIA  category  range. 

Since  for  any  given  applicant  these  characteristics  are  fixed  across  the  alternative  MOS 
within  his/her  job  list,  their  differential  effect  can  only  be  achieved  in  our  modeling  approach  by 
using  MOS  alternative-specific  weights  in  the  utility.  However,  given  the  large  number  of  MOS 
alternatives,  varying  parameters  for  each  MOS  is  computationally  prohibitive.  Additionally,  a 
number  of  these  parameters  are  not  likely  to  vary  substantially  since  many  MOS  share  common 
characteristics.  For  these  reasons,  we  only  used  10  weights  for  each  applicant  characteristic; 
nine  weights  specific  to  the  aptitude  areas  for  alternatives  representing  Army  MOS,  plus  another 
weight  for  the  alternative  representing  non-accession.  An  exception  was  AFQT  percentile  score, 
for  which  we  only  specified  a  non-zero  weight  for  the  non-accession  alternative. 


Specifying  the  Random  Utility  Distributional  Assumptions 

In  addition  to  specifying  the  deterministic  utility  function,  we  needed  to  specify  the 
distributional  assumptions  about  the  random  errors  Ey  s  in  applicant  utilities,  which  in  full  are 

given  by 


^  r/t,  +F,(z)+r,(A',c)+£:,.,  ;  =  1.2... .,101; 

*  +£„.».  2=999. 

Doing  so  would  fully  describe  the  structure  of  the  applicant’s  utilities.  As  in  most  analyses,  we 
assumed  that  training  choice  observations,  and  by  implication  the  Ey  s,  across  applicants  are 

statistically  independent.  For  the  intra-individual  correlation  structure,  the  Ey  s  are  usually 
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assumed  to  be  independent  and  identieally  distributed  as  a  Type  I  extreme  value  distribution.^ 
However,  the  latter  assumption  raises  two  implications  that  are  difficult  to  justify.  We  describe 
the  issues  below  and  provide  an  alternative  error  structure  specification. 

First,  the  assumption  states  that  the  variance  of  ,  which  represents  our  uncertainty,  is 

the  same  for  both  MOS  and  non-accession  alternatives.  This  assumption  was  difficult  to  justify. 
For  one,  the  extent  of  our  knowledge  of  the  applicant’s  utility  is  different  for  non-accessions 
compared  to  MOS  alternatives,  as  indicated  by  the  difference  in  their  systematic  components 
shown  in  the  equation  above.  Equally  as  important,  we  were  dealing  with  two  different  types  of 
alternatives,  specifically,  military  jobs  and  civilian  jobs.  Therefore,  to  account  for  potential 
differences  in  error  variance  of  random  utility,  we  specified  a  common  scale  for  MOS 
alternatives  and  a  different  scale  for  the  non-accession  alternative.^ 

Second,  the  usual  distributional  assumption  regarding  E^j  also  implies  that  the  error 

variance  of  utility  is  the  same  across  individual  recruits  for  a  given  alternative  (i.e.,  errors  are 
homoscedastic).  This  is  separate  from  the  first  issue  above,  which  compares  error  variance 
between  alternative  utilities.  It  is  of  special  concern  in  relation  to  the  more  than  20  percent  of 
total  applicants  with  job  “lists”  consisting  of  a  single  MOS  opportunity,  since  EPAS  is  not 
expected  to  achieve  direct  classification  efficiency  from  these  individuals.  Failing  to  address 
potential  heterogeneity  in  error  variance  of  single-  and  multiple-opportunity  applicants’  utility 
could  lead  to  over-  or  under-estimation  in  the  remaining  80  percent  of  the  applicant  population. 
To  account  for  this  heterogeneity  we  included  a  scale  factor  for  single-opportunity  applicants, 
thereby  yielding  a  different  error  variance  compared  to  that  of  multiple-opportunity  applicants.^ 

In  sum,  the  following  are  the  final  distributional  assumptions  for  the  random  variable  E^j 

adopted  for  our  JCM  after  adding  the  two  error  variance  modifications.  These  assumptions 
completed  the  mathematical  structure  of  the  applicant’s  utility. 

^  The  Type  I  extreme  value  distribution  with  scale  parameter  (7  has  cumulative  distribution  function 

F{£^  =  exp{—  exp[—  cr^]} .  This  distribution  has  a  mean  of  zero  and  variance  71^  Note  that  the  scale 

parameter  is  inversely  proportional  to  the  error  variance. 

*  The  scale  parameter  for  MOS  alternatives  was  specified  a  priori  to  be  greater  than  or  equal  to  that  for  non¬ 
accession.  While  this  relationship  was  more  of  a  constraint  in  our  modeling  approach,  it  can  be  justified  for  two 
reasons.  First,  the  systematic  utility  for  MOS  alternatives  includes  more  observed  information  than  that  for  non¬ 
accession.  Second,  the  utility  for  non-accession  alternative  represents  diverse  civilian  jobs  while  that  for  MOS 
alternatives  represent  specific  military  jobs.  These  two  observations  will  have  the  effect  of  making  the  error  variance 
of  random  utility  for  MOS  alternatives  lower  compared  to  that  for  the  non-accession  alternative,  or,  equivalently, 
making  the  MOS  scale  parameter  greater  than  that  for  the  non-accession  alternative. 

’’  Two  types  of  individual  who  are  likely  involved  in  single-opportunity  transactions  are:  (1)  applicants  who  are 
eligible  for  one  or  two  MOS  only;  and  (2)  applicants  who  seek  specific  MOS,  often  regardless  of  monetary 
incentives.  An  inspection  of  the  data  appears  to  support  these  two  cases.  Out  of  the  top  five  MOS  involved  in  single¬ 
opportunities,  three  have  relatively  low  minimum-standards,  specifically:  1 IX  (Infantry),  92G  (Food  Service 
Operations),  and  88M  (Motor  Transport  Operator).  The  other  two  MOS  are  95B  (Military  Police)  and  91 W  (Health 
Care  Specialist),  which  respectively  have  moderate  to  high  minimum-standards  and  appear  to  be  sought  after  despite 
having  very  low  monetary  incentives.  The  first  two  MOS  involved  applicants  whose  training  choices  reflect 
decisions  made  with  limited  options.  The  other  three  MOS  involved  applicants  whose  training  choice  decisions  are 
based  heavily  on  specific  factors.  The  utilities  of  applicants  in  both  cases  possibly  are  not  represented  as  well  by  the 
systematic  utility  V.- ,  thereby  yielding  relatively  larger  error  variance. 
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(1)  The  EyS  are  independent  across  individual  applicants. 

(2)  The  distribution  ofEy ’s  is  Type  I  extreme  value  with  variance  that  is 
characterized  by  the  positive  constants  S  and  A.  as  follows.  The  variance  of  the 
non-accession  alternative  is  A}  times  that  of  an  MOS  alternative,  and  the  variance 
for  applicants  with  multiple  opportunities  is  times  that  for  applicants  with  a 
single  opportunity. 

(3)  For  the  /th  applicant  with  multiple  opportunities,  the  variance  of  Ey  ’s  is 

{SttY  )  for  MOS  alternatives  (j=l,2,...,101)  and  16  for  the  non¬ 
accession  alternative  (j=999). 

(4)  For  the  zth  applicant  with  a  single  opportunity,  the  variance  of  Ey ’s  is  /[gA^  ) 

for  MOS  alternatives  (j=l,2,...,101)  and  / 6  for  the  non-accession  alternative 
0=999). 

(5)  Random  errors  associated  with  MOS  alternatives  (£,, ,  £,2 , . . . ,  )  are 

independent  of  the  random  error  for  the  non-accession  alternative  (.£,999 ) . 

(6)  Conditional  on  the  /th  applicant  joining  the  Army,  the  random  errors 
(£,, ^iio, )  are  independent. 


Probabilistic  Job  Choice  Model 

Having  specified  the  deterministic  utility  function  and  the  distributional  assumptions  of 
the  residual  utility  term,  we  now  specify  the  full  form  of  the  applicant’s  probability  choice  model 
separately  for  multiple-  and  single-opportunity  applicants.  To  keep  our  expressions  compact,  we 
employ  the  following  substitutions  in  our  model  formulas: 


V,j(Z,.X„,C,}=  A,+V,^(Z)+V„{X.C) 


V„„{Z,)=A 


+v 

999  ^  '  /999 


iz) 


Multiple-Opportunity  Applicant.  Given  that  applicants  are  expected  to  behave  rationally 
and  that  their  utilities  are  given  by  equation  (2)  with  the  aforementioned  distributional 
assumptions,  then  the  training  choices  of  applicants  can  be  described  by  the  Nested  Logit  (NL) 
probability  model,  with  MOS  alternatives  forming  one  nest  and  the  non-accession  alternative 
constituting  a  nest  of  its  own.  Mathematically,  this  can  be  represented  as  follows. 


Without  loss  of  generality,  suppose  that  the  job  list  of  the  /th  applicant  is  comprised  of 
the  MOS  alternatives  labeled  j  =  1,2,..., zn ,  then  the  probability  that  s/he  chooses  the  Ath  MOS 
alternative  can  be  given  by 
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P,(k) 


exp[«  K„  (Z, ,  X. ,  C, )]  X  X  exp[ai  V,  (z, ,  X„ ,  C, )] 


exp[^,,,  (Z, )]  +  exp[^A  P:.  (z, ,  X, ,  C, )] 


while  the  probability  that  s/he  chooses  not  to  join  the  Army  can  be  given  by 

exp[<^999(Z,.)] 


pX999)  = 


exp[^,999  te  )]  +  E  exp[^Z  V,^  (z,. ,  JT, ,  C,. )] 


The  NL  model  form  describes  or  “estimates”  training  choice  by  providing  a  probability 
that  the  zth  applicant  with  characteristics  Z, ,  who  is  being  processed  by  a  guidance  counselor 

with  performance  measure  C, :  (1)  chooses  the  Alh  MOS  alternative  among  m  alternatives  with 
attributes  or  (2)  decides  not  to  join  the  Army.  This  probability  has  an 

“average”  interpretation  from  the  researcher’s  point  of  view.  That  is,  over  many  samples  of 
applicants  with  the  same  characteristics  and  the  same  alternatives  and  attributes,  the  probability 
represents  the  proportions  of  applicants  with  specified  characteristics  who  chose  a  given 
alternative. 

Alternatively,  the  model  can  also  be  expressed  in  sequential  probability  form,  which 
describes  an  applicant’s  decisions  using  two  “levels”:  (1)  to  join  (or  not  join)  the  Army;  and  (2) 
MOS  choice  if  joining  the  Army.  It  can  be  verified  algebraically  that  for  alternatives 
k  =  1,2,..., w  the  choice  probability  is 

(a)  =  [l  -  (999)] X  (A:  1  join  Army). 


where 


\  exp[^ZK,(Z,,A'„,C,)] 

P,  (a  I  join  Army)  =  — - — — . 

y;exp[<Sif^,(z,,X„,C,)] 

7=1 

The  factor  [l  -  P^  (999)]  is  just  the  probability  that  the  applicant  will  join  Army.  The  factor 
^,(^1  join  Army)  is  the  probability  that  he  will  choose  the  Ath  MOS  alternative  among  m 
alternatives  conditional  on  his/her  joining  the  Army.  The  form  of  P^k  \  join  Army)  is  also 
known  as  the  Multinomial  Logit  (MNL)  model. 
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Single-Opportunity  Applicant.  The  probability  choice  model  for  single-opportunity 
applicants  is  different  in  form  but  consistent  with  the  above  models  for  multiple-opportunity 
applicants.  For  the  zth  applicant  with  only  one  option  represented  by  the  ^h  MOS  alternative, 
the  probability  for  choosing  the  hh.  MOS  over  non-accession  is  given  by 

P  fjV  exp[F„fe.A-..C,)] 

''  exp[K,„,(z,)]+exp[K.(Z„jr.,C,)] 

The  probability  for  not  joining  the  Army  is  simply  (999)  =  1  -  (A:).  This  simpler  form  is  just 

the  logistic  probability  model. 

Taken  together,  the  different  probability  functions  above  represent  the  major  components 
of  a  single,  probabilistic  JCM.  The  constants,  weights,  and  scale  parameters  for  the 
accession/non-accession  and  MOS  choice  components  of  the  model  combine  to  characterize  the 
JCM  and  must  be  jointly  estimated.  Because  of  the  large  number  of  model  parameters,  we 
employed  a  two-stage  estimation  strategy  by  taking  advantage  of  the  two-level  sequential 
probability  form  of  the  model.  The  first  stage  involved  estimating  the  parameters  in  the 
conditional  MNL  model  {k  \  join  Army)  using  an  iterative  process  (detailed  in  the  next  part  of 

the  report).  Only  multiple-opportunity  applicants  were  used  in  this  stage.  In  the  second  stage,  we 
estimated  all  parameters  in  the  combined  two-level  model,  including  those  associated  with  the 
decision  to  join  or  not  join  the  Army,  using  both  single-  and  multiple-opportunity  applicants. 
Parameter  estimates  obtained  in  the  first  stage  were  used  as  starting  values  in  the  second  stage. 


ESTIMATING  AND  EVALUATING  THE  JOB  CHOICE  MODEL  (JCM) 

Having  constructed  the  JCM,  we  moved  to  estimate  its  major  components  for  use  in  the 
EPAS  Simulation.  In  the  following  sections,  we  detail  our  procedure  for  estimating  these 
components.  Most  important,  we  document  and  discuss  the  results  of  an  empirical  evaluation  of 
the  model’s  accuracy  for  simulating  applicants’  actual  job  choices. 

Estimation  Samples 

All  JCM  estimates  were  based  on  Army  applicant  and  accession  data  covering  Fiscal 
Year  (FY)  2002.  For  modeling  and  estimation  purposes,  we  partitioned  the  FY  2002  into  four 
quarters  based  on  the  effective  dates  of  the  EB/ACF  incentive  packages  produced  by  the  Army’s 
quarterly  Enlistment  Incentive  Review  Board  (EIRB)  meetings.  Separate  models  were 
estimated  by  EIRB  quarter,  using  the  cut-off  dates  and  sample  sizes  shown  in  the  table  below. 
Note  that  calendar  and  EIRB  quarters  closely  match  but  do  not  coincide.  In  particular,  the  first 
quarter  starts  later  than  the  actual  effective  date  of  1 1  September  2001,  while  the  last  quarter 
continues  for  one  more  week  past  the  EIRB  end-date  of  23  September  2002. 


®  The  EIRB  is  comprised  of  policy  representatives  from  G-1,  USAREC,  and  HRC.  They  review  the  existing 
incentive  structure  vis-a-vis  MOS  fill  to-date  and  training  seat  availability,  and  may  recommend  incentive  changes 
aimed  toward  making  MOS  targets  at  least  cost. 
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Table  2.  JCM  Estimation  Quarters  and  Sample  Sizes 


Quarter 

Start  Date 

End  Date 

Total  Size 

Sample  Size 

1 

October  1,  2001 

December  3,  2001 

14,236 

4,085 

2 

December  4,  2001 

March  3,  2002 

22,049 

4,390 

3 

March  4,  2002 

June  2,  2002 

24,264 

4,395 

4 

June  3,  2002 

September  30,  2002 

32,407 

4,421 

A  few  alternatives  were  relatively  large  compared  to  the  others.  For  example,  1 IX  and 
non-accessions  accounted  for  over  30%  of  all  applicants,  while  fewer  than  10  out  of  a  total  of 
1 02  MOS  alternatives  accounted  for  more  than  50%  of  all  applicants.  Because  of  the  potential 
for  under  (and  over)  representation,  we  employed  a  choice-based  sampling  strategy  to  ensure  that 
all  MOS  alternatives  were  adequately  represented  in  the  estimation  sample.  To  carry  out  this 
sampling  strategy,  we  first  grouped  applicants  according  to  their  chosen  MOS  (or  non¬ 
accession).  Applicants  were  then  selected  by  under-sampling  from  the  larger  MOS  groups  and 
over-sampling  from  the  smaller  MOS  groups.  To  ensure  that  our  sampling  strategy  did  not 
artificially  bias  JCM  estimates,  we  assigned  weights  to  applicants  during  model  estimation  that 
were  equal  to  the  reciprocal  of  the  sampling  rates  in  their  respective  MOS  groups. 


Data  Preparation  and  Simplification 

Information  on  applicants’  attributes  and  job  opportunities  represent  real-world  data. 
Because  of  this,  and  as  is  characteristic  of  most  non-experimental  data,  certain  features  of  the 
data  were  not  directly  amenable  to  our  modeling  approach  without  simplification  and/or 
restructuring.  For  example,  some  applicants  had  more  than  one  search  date,  indicating  that  they 
had  visited  the  MBPS  multiple  times  before  making  a  final  decision  or  at  some  point  changed 
their  mind.  While  having  multiple  search  dates  reflects  a  legitimate,  real-world  property  of  the 
data,  such  a  feature  does  not  lend  itself  to  a  straightforward  and  parsimonious  application  of  our 
approach  without  greatly  increasing  (unnecessarily)  the  complexity  of  the  JCM.  Therefore,  to 
minimize  potential  problems  with  the  data,  we  made  several  simplifications  and  modifications  to 
the  data  prior  to  estimating  the  JCM.  In  all  cases,  our  goals  in  preparing  and  simplifying  the  data 
were  twofold:  first,  to  retain  meaningful  real-world  features  of  the  data  to  ensure  the  realism  and 
generalizability  of  the  EPAS  Simulation,  and  most  importantly,  its  results;  and  second,  to 
increase  the  reliability  of  the  data  to  ensure  that  it  met  basic  model  requirements  so  as  to  produce 
a  meaningful  (and  accurate)  evaluation  of  the  JCM. 

Specifically,  there  were  four  major  features  of  the  data  requiring  attention.  These  were: 
(1)  identifying  applicants’  search  dates  for  use  in  the  JCM  estimation  and  EPAS  simulation,  as  a 
number  of  applicants  had  multiple  search  dates;  (2)  including  or  excluding  applicants  with  job 
opportunity  lists  consisting  of  a  single  opportunity,  as  these  applicants’  choice  behavior  (and 
preferences)  were  likely  different  from  those  of  most  applicants;  (3)  aggregating  applicants’  job 
opportunity  lists,  as  a  sizeable  percentage  of  applicants  made  multiple  queries  (within  the  same 
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search  date);  and  (4)  configuring  the  job  choice  space.  In  the  following  subsections,  we  briefly 
summarize  each  issue  and  how  each  was  handled  and  why. 

Identifying  Applicants  ’  Search  Dates  for  JCM  Estimation  and  EPAS  Simulation 

Inspection  of  the  data  indicated  that  there  were  a  number  of  applicants  with  multiple 
search  dates.  Consequently,  we  were  faced  with  the  issue  of  how  to  handle  applicants  with 
multiple  search  dates.  While  this  usually  represented  a  legitimate  feature  of  the  data  (and  not  a 
data  entry  error),  as  there  are  applicants  who  make  multiple  visits  to  the  MEPs  before  making  a 
final  job  choice  decision,  the  inclusion  of  multiple  search  dates  into  the  JCM  would  have  greatly 
added  to  the  complexity  of  the  model.  To  simplify  the  model,  we  elected  to  use  the  search  date 
matching  the  applicants’  reservation  date.  There  were  two  reasons  for  this:  first,  because  the 
search  date  from  which  the  actual  job  choice  was  made  represents  the  choice  space  most 
proximal  to  an  applicant’s  actual  decision,  it  should  exert  the  greatest  influence  on  their  job 
choice;  and  second,  including  the  most  recent  search  date  already  captures  the  “effects”  of 
previous  search  dates.  That  is,  it  is  reasonable  to  expect  that  opportunities  presented  on  the  final 
search  date  are  largely  a  function  of  applicants’  preferences  and  choice  behavior  manifested 
during  earlier  search  dates.  Therefore,  including  past  search  date  information  is  not  likely  to  add 
appreciably  to  the  JCM  estimation,  since  its  “effects”  are  transmitted  through  the  most  recent 
search  date. 

Applicants  with  Single  Opportunity  Job  Lists 

One  issue  that  we  faced  at  the  beginning  was  how  to  deal  with  applicants  with  a  single 
job  opportunity,  whether  to  exclude  or  include  them  in  model  estimation.  A  substantial 
percentage  of  applicants  (roughly  20%)  had  job  opportunity  lists  consisting  of  a  single 
opportunity.  There  are  two  reasonable  explanations  for  this:  (1)  that  these  applicants  came  to  the 
MEPS  with  well-defined  preferences  and  were  interested  in  a  specific  job  (MOS);  or  (2)  that 
these  applicants  were  eligible  for  a  limited  number  of  MOS  only  (e.g.,  female  applicants  with 
low  AFQT  scores).  These  applicants  contribute  to  the  first-level  of  the  JCM  (i.e.,  join  or  not  to 
join  the  Army)  but  not  directly  in  the  second-level  as  their  MOS  is  “fixed”  once  they  decide  to 
join  the  Army.  We  included  these  applicants  in  our  model  but  treated  them  differently  from 
applicants  with  multiple-opportunities.  (Technical  details  were  provided  earlier  in  our 
mathematical  description  of  the  JCM.)  There  were  two  reasons  for  this.  First,  because  the 
preferences  and  choice  behavior  of  applicants  with  single  job  opportunities  likely  differ  from  that 
of  most  applicants.  For  instance,  whereas  most  applicants’  decisions  to  access  or  not  access  will 
partly  be  a  function  of  the  relative  attributes  of  the  available  job  opportunities,  these  applicants’ 
decisions  are  likely  determined  mainly  by  the  simple  (un)availability  of  the  preferred 
opportunity.  Second,  while  these  single  opportunity  applicants  are  beyond  the  reach  of  EPAS 
optimization,  they  contribute  to  the  entire  Army  accession  cohort  and  therefore  impact  the  EPAS 
model.  The  “first  case”  applicants  clearly  hold  strong  and  well-defined  preferences,  and  EPAS  is 
not  likely  to  exert  much  of  an  influence  on  their  job  choice  decisions.  In  sum,  these  applicants 
are  part  of  real-world  REQUEST  and  excluding  them  or  “fixing”  their  choices  biases  the  model 
and  field  test  evaluation. 

Aggregating  Applicants  ’  Job  Opportunity  Lists 

Many  applicants  made  multiple  queries  within  the  same  search  date.  This  raised  the 
question  of  how  to  deal  with  multiple  queries,  which  produced  multiple  opportunity  lists  whose 
similarity  in  MOS-reception  dates  varied.  Comparable  to  multiple  search  dates,  while  this 
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feature  represents  a  legitimate  facet  of  the  data,  its  inclusion  would  have  added  significant 
complexity  to  the  model.  To  simplify  this  data  feature,  we  elected  to  aggregate  opportunities 
across  queries  (within  the  same  search  date),  dropping  duplicate  MOS-reception  date 
opportunities  and  reordering  the  rank  order  of  opportunities  based  on  the  combined  list.  This 
approach  was  taken  for  several  reasons.  The  first  reason  is  that  all  opportunities  presented  define 
the  choice  space  for  the  applicant.  For  example,  applicants  may  make  a  job  choice  at  any  time, 
irrespective  of  whether  the  opportunity  appears  in  the  current  query.  While  modeling  multiple 
queries  unnecessarily  increases  model  complexity,  excluding  opportunities  artificially  truncates 
the  choice  space  and  limits  a  meaningful  real-world  feature  of  the  data,  as  it  ignores 
opportunities  considered  by  applicants  when  making  their  decisions.  This  in  turn  potentially 
biases  JCM  probability  estimates  of  applicant  choices,  which  are  conditional  on  the  full  set  of 
alternatives  (and  not  an  artificially  defined  subset).  A  second  reason  for  the  approach  selected  is 
that  Army  counselors,  who  potentially  play  a  significant  role  in  applicants’  job  choice  decisions, 
are  likely  to  treat  multiple  queries  as  a  single  query.  That  is,  counselors  are  normally  familiar 
with  the  Army’s  prioritization  of  job  opportunities.  Because  they  are  incentivized  to  do  so, 
counselors  can  reasonably  be  expected  to  “sell”  applicants  on  the  highest  priority  job(s), 
irrespective  of  the  particular  query  said  job(s)  appear  in.  Therefore,  excluding  opportunities 
could  further  bias  JCM  estimates  (positively  or  negatively)  because  it  minimizes  an  important 
effect,  counselor  performance,  which  meaningfully  contributes  to  applicants’  choice  behavior. 


Configuring  the  Job  Choice  Space 

Inspection  of  applicants’  job  (MOS)  choices  indicated  that  across  the  full  fiscal  year  there 
were  upwards  of  155  total  MOS  to  select  from.  While  including  all  possible  jobs  may  be  ideal 
from  a  conceptual  standpoint,  practically  this  represented  a  considerably  large  choice  space  to 
define  computationally  and  model  accurately.  To  address  this,  we  elected  to  reduce  the  choice 
space  by  combining  jobs  (MOS),  specifically  jobs  with  small  sample  sizes  («),  that  were:  (a) 
similar  in  job  content  (i.e.,  were  members  of  the  same  Career  Management  Field  or  Aptitude 
Area);  and  (b)  similar  in  their  incentive  profiles.  There  were  several  reasons  recommending  this 
approach.  First,  reducing  the  choice  space  would  minimize  potential  estimation  problems 
resulting  from  the  increased  dimensionality  (and  complexity)  of  the  original  choice  space. 
Second,  increasing  the  n  of  these  jobs  increases  the  accuracy  of  the  model  estimates.  Both 
minimize  the  possible  bias  in  JCM  estimates  without  sacrificing  important  information,  as  the 
jobs  combined  represent  jobs  with  similar  attributes  and  therefore  are  likely  to  elicit  similar 
preferences  (utilities)  from  applicants.  When  done,  the  final  job  choice  configuration  consisted 
of  101  MOS  alternatives,  which  across  the  four  FY  quarters  represented  99%-100%  of  the 
original  job  choice  space  and  resulted  in  a  doubling  (on  average)  of  the  median  n.  These  MOS 
alternatives  are  reported  in  Table  A.l. 


Estimation  Procedure 

Consistent  with  our  design,  JCM  estimates  were  generated  by  FY  quarter  using  the 
applicable  sample.  For  each  quarter,  the  estimation  procedure  consisted  of  the  same  two-stage 
process,  as  follows; 
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Stage  One:  Estimating  the  Second-Level  of  the  JCM.  In  the  first  stage,  estimates  for  the 
JCM  were  computed  for  the  second-level  of  the  model,  applicants’  job  opportunity  choice,  using 
the  multinomial  logit  model  (MNL)  described  in  the  preceding  part  of  the  report.  Estimating  the 
MNL  followed  a  multi-step,  iterative  process.  In  the  first  step,  a  main  effects  MNL  was 
estimated.  This  main  effects  MNL  consisted  of  multiple-opportunity  applicants  and  MOS 
alternative-specific  attributes,  and  counselor  performance,  as  documented  in  the  preceding  part 
of  the  report.  After  estimating  this  main  effects  model,  we  evaluated  overall  model  fit  and  fit  by 
the  different  subgroups  and  job  types  (or  MOS  aptitude  areas)  to  ensure  that  the  second-level 
JCM  reasonably  predicted  applicants’  actual  choices.  In  all  cases,  this  evaluation  indicated  that 
model  fit  at  the  subgroup  (i.e.,  gender,  education  status,  etc.)  and  job  type  (i.e..  Clerical,  Combat, 
etc.)  levels  could  be  significantly  improved  by  adding  selected  interaction  terms  to  the  MNL. 

The  purpose  of  interaction  terms  is  to  capture  meaningful  differences  in  MOS  preferences  by 
applicant  characteristics  (e.g.,  gender  differences  in  preferences  for  Clerical  jobs,  such  that 
females  tend  to  prefer  these  jobs  more  than  male  applicants).  Using  fit  diagnostics  at  the 
subgroup  and  job  type  level  as  a  guide,  interaction  terms  were  selectively  added  to  the  MNL. 
This  continued  until  fit  diagnostics  at  the  subgroup  and  job  type  levels  met  desirable  levels  of  fit, 
at  which  point  the  model  estimation  process  was  stopped.  As  will  be  demonstrated  in  the  next 
section,  adding  interaction  terms  produced  substantial  increments  in  fit,  particularly  at  the 
subgroup  and  job  type  level. 

Stage  Two:  Estimating  the  First-Level  of  the  JCM.  In  the  second  stage,  estimates  were 
additionally  computed  for  the  first-level  of  the  JCM,  applicants’  decision  to  join  (or  not  join)  the 
Army,  using  the  nested  logit  model  (NL),  also  summarized  previously.  The  parameters  in  the 
first-level  JCM  were  re-estimated  in  this  stage,  using  estimates  obtained  in  the  Stage  One  as 
starting  values.  The  parameter  estimates  obtained  in  this  stage  specify  the  full  two-level  JCM 
estimate.  The  second  stage  estimation  was  not  carried  out  until  the  MNL  (at  the  first  stage) 
demonstrated  desirable  (and  comparable)  levels  of  model  fit  both  overall  and  across  the  different 
subgroups  and  job  types.  Because  the  increased  complexity  of  the  NL  models  increases  the 
computational  time  and  resources  required  for  estimation,  estimation  proceeded  iteratively  until: 
(1)  a  desirable  level  of  model  fit  was  obtained;  and  (2)  subsequent  iterations  failed  to  produce 
significant  increments  in  model  fit,  either  overall  or  at  the  subgroup  and  job  type  level.  When 
these  criteria  were  met,  NL  estimation  was  halted  even  if  the  model  had  not  fully  converged. 

Details  on  observed  fit  for  the  MNL  and  NL  models  (by  quarter)  and  specific  criteria 
used  are  documented  further  in  the  next  section.  All  JCM  estimates  for  both  the  MNL  and  NL 
were  computed  using  the  BIOGEME  software  (Bierlaire,  Bolduc,  &  Godbout,  2003). 


Estimation  Results  and  Fit  Diagnostics 

The  JCM  parameter  estimates  obtained  at  the  end  of  the  two-stage  estimation  procedure 
are  presented  in  Appendix  B.  The  alternative-specific  constants  for  each  MOS  are  reported 
Table  B.l .  The  B-  and  G-weights  in  the  deterministic  utility  and  scale  parameter  for  the  random 
utility  are  reported  in  Table  B.2.  To  facilitate  comparisons  across  quarters,  rescaled  versions  of 
the  estimates  are  reported  in  Tables  2  and  3  for  the  first  20  MOS  alternatives  and  non-accession 
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constants,  coefficients  for  all  alternative-specific  attributes  and  selected  applicant  characteristic.^ 
Model  fit  diagnostics  are  reported  in  Appendix  C. 

Utility  Parameter  Estimates 

As  ean  be  seen  from  Table  B.l,  across  all  quarters,  most  MOS  alternative-specific 
constants  are  negative,  reflecting  the  fact  that,  on  average,  most  MOS  are  associated  w^ith  lower 
utilities  compared  to  1 IX  (Infantry). This  makes  sense  since  1 IX:  (1)  is  open  to  virtually  all 
recruits,  particularly  low-aptitude  recruits  with  fewer  job  choices;  (2)  tends  to  be  highly 
incentivized  to  attract  large  number  of  applicants;  and  (3)  is  always  a  high  priority  (or  highly 
ranked)  MOS.  Consequently,  the  probabilities  of  applicants  joining  1 IX  tend  to  be  higher,  on 
average,  than  those  for  other  MOS.  The  only  MOS  for  which  this  is  consistently  not  the  case 
across  all  four  quarters  is  98X  (EW/SIGINT  Specialist-Linguist).  Similarly,  these  differences 
between  1 IX  and  the  other  MOS  tend  to  be  statistically  significant  (see  “T-stat”  column; 
significant  differences,  <  .05  are  bolded).  On  average,  roughly  10%  of  the  other  MOS  per 
quarter  display  baseline  utilities  comparable  to  1 IX.  Alternative-specific  constants  are  also 
generally  comparable  across  four  quarters  in  terms  of  their  rank  ordering. 

Turning  to  Table  4,  among  alternative-specific  attributes,  those  that  consistently  exhibited 
significant  effects  on  applicant  choices  across  quarters  are:  (1)  rank  order  of  the  MOS;  (2) 
counselor  performance;  (3)  SB  incentive;  and  (4)  AA  score.  Estimates  of  rank  order  coefficient 
are  consistently  negative  and  statistically  significant  for  all  quarters.  Because  alternatives  at  the 
top  of  the  job  list  have  lower  numeric  rank  order  values,  it  is  important  for  this  parameter  to  be 
negative  for  EPAS  to  have  a  positive  impact  on  REQUEST.  However,  as  described  by  equation 
(1),  the  overall  weight  of  rank  order  is  dependent  on  the  performance  of  the  counselor  processing 
the  applicant,  which  has  positive  significant  coefficient  across  quarters.  The  combined  effect  of 
this  interaction  is  that  the  potential  positive  impact  of  EPAS  on  REQUEST  can  be  expected  from 
better-performing  counselors  but  not  from  counselors  performing  poorly. 

Among  the  monetary  incentives,  only  SB  consistently  exerted  a  positive,  significant 
effect  on  applicants’  job  choices  across  all  quarters.  The  positive  SB  coefficient  estimates  can  be 
interpreted  to  mean  that  the  incentive  was  effeetive  in  making  near  term  training  class  seats 
attractive  to  applicants.  The  interaction  between  SB  incentive  and  high  school  senior  education 
status  is  significantly  negative  for  the  third  quarter,  but  not  significant  for  the  other  three 
quarters.  This  is  not  surprising  given  that  seniors  generally  would  not  be  able  to  access  near  term 
MOS  alternatives  during  the  third  quarter,  which  would  be  around  the  last  three  months  of  the 
school  year  (i.e.,  March,  April,  and  May).  The  results  for  the  other  monetary  incentives  are 
mixed.  The  TOS+EB+ACF  composite  utility  (see  pp.  8-9)  has  a  positive  significant  effect  in  the 
fourth  quarter,  a  not  significant  (but  somewhat  substantial)  positive  effect  in  the  first  quarter,  and 
not  significant  negative  effect  in  the  second  and  third  quarters.  The  AB  incentive  has  positive 
significant  effect  in  the  first  quarter,  not  significant  but  non-negligible  effect  in  the  second  and 


°  B-  and  G-weights  are  reported  “relative”  to  the  scale  of  the  utilities  for  single-opportunity  applicants.  To  obtain 
parameters  relative  to  the  majority  of  applicants  with  multiple  opportunities,  estimates  corresponding  to  MOS 
alternatives  were  multiplied  by  LAMDA  and  DELTA  estimates  while  those  corresponding  to  the  non-accession 
alternatives  (suffix  by  999)  were  multiplied  by  DELTA. 

Readers  are  reminded  not  to  interpret  the  absolute  level  of  the  constants  to  mean  that  the  utilities  for  most  MOS 
are  negative,  as  these  values  are  not  average  applicant  utilities.  These  constants  reflect  the  standing  of  an  MOS 
relative  to  1 IX  and  therefore  represent  differences  in  (average)  utility  between  an  MOS  and  1 IX. 
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third  quarter,  and  not  significant  negative  effect  in  the  fourth  quarter.  The  HG  incentive  has  a 
substantial  but  not  significant  negative  effect  in  the  last  three  quarters.  This  appears  not 
surprising  given  that  an  intended  policy  goal  of  the  incentive,  to  make  the  Army  attractive  to 
college  individuals,  has  already  taken  effect  in  our  recruit  data." 

Finally,  the  applicant’s  AA  scores  for  MOS  alternatives  in  the  job  list  have  a  positive 
significant  effect  across  quarters,  demonstrating  that  applicants  tend  to  choose  the  MOS  training 
opportunity  for  which  they  display  the  highest  AA  score.  This  observation  has  an  important 
implication  for  EPAS.  It  suggests  an  existing  positive  person-job-match  in  REQUEST 
transactions,  which  was  assumed  in  the  EPAS  model  to  be  random.  Consequently,  for  EPAS  to 
have  a  significant  impact  on  REQUEST,  its  effect  would  have  to  be  greater  than  that  needed  if 
the  person-job-match  were  in  fact  random  (i.e.,  AA  weights  are  not  significantly  different  from 
zero). 


As  for  applicant  attributes  (gender,  education  status,  AFQT  category  and  percentile  score, 
and  geographic  region),  there  were  applicant  differences  associated  with  enlistment  in  the  Army 
and  in  preferences  to  choose  certain  types  of  jobs.  Starting  with  Army  enlistment  (see 
parameters  post-scripted  with  a  “999”),  overall,  none  of  the  applicant  attributes  consistently 
exerted  significant  effects  across  all  four  quarters.  However,  there  were  some  significant 
differences  by  quarter.  For  example,  there  were  significant  gender  differences  (G_sexM999)  in 
utilities  during  the  Second  and  Third  quarters,  such  that  males  (on  average)  exhibited  a  more 
positive  utility  (and  preference)  than  females  to  enlist.  Similarly,  high  school  seniors 
(G_edS999)  were  less  likely  to  join  the  Army  compared  to  high  school  graduates  during  the  first 
three  quarters  when  school  was  still  on  going.  This  is  not  the  case  after  the  end  of  the  school 
year  during  the  fourth  quarter,  as  evidenced  by  a  not  significant  negative  interaction  effect.  Only 
geographic  region  (G_RS999)  did  not  produce  a  significant  effect  on  applicant  enlistment 
preference  at  some  point  during  the  FY.  Shifting  to  applicant  differences  in  MOS  job 
preferences,  applicants  (on  average)  did  differ  in  the  utilities  associated  with  different  types  of 
jobs.  For  example,  males  consistently  tended  to  attribute  (on  average)  greater  utility  to 
Electronic  (G_sexM3),  General  Maintenance  (G_sexM5),  and  Mechanical  Maintenance 
(G_sexM6)  jobs  than  did  females.  This  trend  makes  sense  given  that  historically  male  Army 
applicants  generally  achieve  higher  AA  scores  and  demonstrate  a  greater  propensity  to  enlist  in 
these  jobs  than  females.  Similarly,  consistent  with  the  fact  that  they  tend  not  to  be  eligible  for 
these  types  of  jobs,  non-high  school  graduates  tended  to  attribute  lower  utility  to  Skilled 
Technical  jobs  (G_edNG9)  than  high  school  graduates.  Taken  together,  and  consistent  with  the 
alternative-specific  constants  (discussed  above),  the  direction  and  general  magnitude  of  these 
parameters  are  consistent  with  previous  research  and  observations  of  Army  applicant  job  choices 
indicating  (indirectly)  that  the  model  conforms  with  real-world  job  choice  processes  (and 
utilities). 


"  The  HG  incentive  is  given  to  applicants  with  more  than  30  semester  hours  of  college  if  they  choose  an 
“incentivized”  MOS  (i.e.,  these  are  MOS  with  EB/ACF  incentives).  However,  because  these  MOS  account  for  at 
least  .75%  of  the  101  MOS  alternatives  considered  in  the  JCM,  the  incentive  effectively  functioned  in  the  model  as 
an  indicator  for  college  applicants,  who  tend  to  be  more  selective  and  less  likely  to  access.  Thus,  the  negative  HG 
effect.  If  we  start  with  the  youth  population  (or  market  that  can  be  reached  by  recruiters)  in  our  modeling,  then  we 
will  be  able  to  see  the  real  impact  of  this  incentive  in  encouraging  youth  to  consider  and  join  the  Army,  and 
different  results  may  likely  be  obtained. 
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Table  3.  Selected  Alternative-Specific  Constant  Parameter  Estimates  by  Quarter.  Scaled  for 
Second-Level  Conditional  MNL  Model. 


ID 

MOS 

Estimate 

T-stat 

Estimate 

T-stat 

Estimate 

T-stat 

Estimate 

T-stat 

1 

IIX 

0.000000 

0.00 

0.000000 

0.00 

0.000000 

0.00 

0.000000 

0.00 

2 

12B 

-1.032514 

-2.88 

-1.205904 

-3.67 

-0.956949 

-3.93 

-1.196391 

-4.54 

3 

12C 

-3.648624 

-3.19 

-4.749867 

-2.82 

-3.658656 

-4.86 

-3.456503 

-4.72 

4 

13B 

-1.524655 

-3.32 

-2.591657 

-1.99 

-1.005380 

-3.77 

-1.780787 

-2.86 

5 

13F 

-2.112759 

-3.27 

-3.600447 

-2.35 

-1.572297 

-4.26 

-2 . 510510 

-3.45 

6 

13M 

-2.798734 

-3.32 

-3.613938 

-2.35 

-2.123325 

-4.48 

-3.458444 

-3.98 

7 

13P 

-3.065477 

-3.32 

-3.996978 

-2.45 

-2.312500 

-4.38 

-3.959124 

-4.16 

8 

13R 

-3.872185 

-3.29 

-4.765011 

-2.63 

-2.850153 

-4 . 65 

-3.780163 

-4.11 

9 

13X 

-2.245583 

-3.27 

-3.189995 

-2.21 

-1 . 637832 

-4.28 

-2 . 055024 

-2.56 

10 

14E 

-3.466895 

-3.28 

-4.535784 

-2.59 

-2.117172 

-4.36 

-2.741935 

-3.45 

11 

14  J 

-2.794358 

-3.29 

-4.494932 

-2.58 

-2 . 151144 

-4.45 

-2.528426 

-3.34 

12 

14R 

-3.319357 

-3.34 

-4.255660 

-2.52 

-1  .  833978 

-4.42 

-3.374030 

-3.83 

13 

14S 

-2.691492 

-3.25 

-3.835277 

-2.40 

-1 . 994968 

-4.39 

-2.971097 

-3.54 

14 

14T 

-3.070878 

-3.33 

-4.348678 

-2.55 

-2.488423 

-4.59 

-3.260988 

-3.80 

15 

18X 

NA 

NA 

0.525125 

1.45 

2.226504 

4.03 

2.665860 

4.04 

16 

19D 

-1 .492637 

-3.27 

-1.374866 

-3.69 

-1.055166 

-4.12 

-1.085599 

-4.47 

17 

19K 

-1.619723 

-3.30 

-1.210841 

-3.65 

-1.018792 

-4.24 

-1.283392 

-4.85 

18 

27D 

-0.132010 

-0.26 

-1.309207 

-1.14 

0.540499 

1.13 

-1.335957 

-1.91 

19 

31C 

-0.917883 

-1.51 

-2.512283 

CO 

CD 

«— i 

-1.395378 

-2.67 

-1.775547 

-2 . 63 

20 

31F 

-1.147512 

-2.26 

-2.577982 

-1 . 94 

-0.446152 

-1 . 17 

-1 .457798 

-2.24 

999 

000 

-1 . 009706 

-4.11 

-1.317931 

-4.56 

-0.903310 

-3.60 

-0.102697 

-0.31 
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Table  4.  Utility  Weights  and  Scale  Parameter  Estimates  by  Quarter.  Scaled  for  Second-Level 
Conditional  MNL  Model. 

Parameter 

First  Quarter 

Second  Quarter 

Third  Quarter 

Fourth  Quarter 

Estimate 

T-stat 

Estimate 

T-stat 

Estimate 

T-stat 

Estimate 

T-stat 

B  Rnk 

-0.011977 

-2.97 

-0.007928 

-3.24 

-0.012900 

-3.44 

-0.021405 

-3.57 

B  RrnkC 

0.000386 

3.32 

0.000184 

3.40 

0.000317 

3.78 

0.000497 

3.82 

B  IsTEAb 

0.053072 

1.59 

-0.004273 

-0.23 

-0.031338 

-0.97 

0.096952 

2.80 

B  SBd 

0.077739 

2.78 

0.019529 

1.99 

0.040865 

2.09 

0.160683 

3.28 

B  SBSd 

0.108585 

1.33 

0.068228 

1.17 

-0.231727 

-2.04 

-0.038425 

-0.24 

B  ABd 

0.072682 

2.21 

0.025353 

1.42 

0.061606 

1.54 

-0.044386 

-0.90 

B  HGd 

-0.026097 

-0.93 

-0.032260 

-1.63 

-0.059423 

-1.65 

-0.046162 

-1.57 

B  AA 

0.026985 

2.90 

0.019150 

3.08 

0.045515 

3.59 

0.075155 

5.22 

G  sexM3 

0.801551 

2.67 

0.379305 

2.60 

0.863203 

3.55 

0.627257 

2.90 

G  sexMS 

1.025620 

2.33 

0.986495 

2.81 

2.576515 

3.95 

0.888596 

2.41 

G  sexM6 

0.630517 

2.38 

0.780355 

3.41 

1.004613 

3.77 

0.744886 

2.96 

G  sexM999 

-1.946523 

-1.72 

-2.875824 

-3.97 

-3.077038 

-3.41 

-0.048199 

-0.14 

G  edS999 

4 . 140386 

3.25 

3.857095 

4.55 

2.578470 

2.64 

-0.465246 

-1.24 

G  RS999 

0.649710 

0.67 

-0.225957 

-0.34 

-0.534816 

-0.66 

0.067626 

0.22 

G  edNG9 

-0.823289 

-2.47 

0.000000 

0.00 

-0.658215 

-2.98 

-1.164770 

-3.46 

Model  Fit 

As  for  overall  model  fit,  pseudo-R^  values  for  the  full  JCM  ranged  from  0.17  (Third 
Quarter)  to  0.23  (First  Quarter).  While  interpretation  of  pseudo-R^  values  is  not  as 
straightforward  as  in  traditional  linear  regression,  these  fit  statistics  compare  (very)  favorably  to 
those  obtained  for  other  prediction  problems  in  the  social  and  applied  sciences,  where 
comparable  effects  sizes  tend  to  average  about  .10-.  15.  While  overall  model  fit  is  informative, 
fit  diagnostics  at  the  subgroup  level  are  equally  if  not  more  important,  particularly  those  that 
more  directly  speak  to  expected  predictive  accuracy.  Fit  diagnostics  at  the  subgroup  and  job 
family  levels  (by  quarter)  are  reported  in  Appendix  C  (Tables  C.l.  through  C.4).  As  an 
orientation  to  the  tables,  the  following  is  a  summary  of  their  contents  (by  column): 

•  “Freq”  reports  the  number  of  times  an  MOS  within  the  applicable  job  family  was 
offered  {not  necessarily  selected)  to  an  applicant  within  the  specified  subgroup  during 
that  quarter.  Note  that  the  number  reported  for  the  special  job  family  “Non-Acc”, 
which  represents  the  non-accession  alternative,  reflects  the  total  number  of  applicants 
within  the  specified  subgroup  during  that  quarter. 

•  “Estimate  (E)”  reports  the  estimated  (or  expected)  probability  (based  on  the  JCM) 
that  an  applicant  from  the  applicable  subgroup  that  would  select  an  MOS  from  the 
specified  job  family  during  that  quarter.  For  the  Army  enlistment  decision  (see  “Non- 
Acc”s),  these  numbers  reflect  the  estimated  probability  that  an  applicant  from  the 
applicable  subgroup  will  elect  to  not  join  the  Army  during  that  quarter. 

•  “Actual  (A)”  reports  the  actual  (or  observed)  proportion  of  applicants  within  the 
applicable  subgroup  who  selected  an  MOS  from  the  specified  job  family  during  that 
quarter.  Operationally,  this  figure  reflects  the  number  of  applicants  within  the 
applicable  subgroup  selecting  an  MOS  from  the  specified  job  family  divided  by  the 
number  of  times  it  was  offered  (“Freq”)  during  the  same  quarter.  For  the  Army 
enlistment  decision  (see  “Non-Acc”s),  these  numbers  reflect  the  actual  (or  observed) 
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proportion  of  applicants  from  the  applicable  subgroup  that  did  not  elect  to  join  the 
Army  during  that  quarter. 

•  “Diff(A-E)”  reports  the  raw  differences  between  the  actual  [“Actual(A-E)”]  and 
estimated  [“Estimate(E)”]  figures  by  subgroup  and  job  family  within  that  quarter. 
Negative  differences  reflect  over-prediction,  while  positive  differences  reflect  under¬ 
prediction.  The  closer  these  differences  are  to  zero  the  greater  the  JCM’s  aecuracy  in 
predicting  applicants’  job  choices. 

•  “Ratio(E/A)”  reports  the  ratio  of  the  estimated  [“Estimate(E)”]  and  actual 
[“Actual(A)”]  figures  by  subgroup  and  job  family  within  that  quarter.  This  diagnostic 
provides  information  comparable  to  “Diff(A-E)”,  but  facilitates  comparisons  across 
subgroups  and  job  families.  By  formulating  these  differences  as  a  ratio  (a 
proportion),  this  diagnostic  controls  for  MOS  and  job  family  differences  in  selection 
proportions —  the  magnitude  of  “Diff(A-E)”  values  is  bigger  for  MOS  opportunities 
that  are  selected  more  frequently.  Ratios  greater  than  1 .00  are  indicative  of  under¬ 
prediction,  while  ratios  less  than  1.00  indicate  over-prediction.  The  closer  the  ratio  is 
to  1 .00  the  more  accurate  the  JCM’s  predictions  of  applicants’  job  choices.  In 
evaluating  (and  diagnosing)  the  intermediate  and  final  forms  of  the  JCM,  we  aimed 
for  ratios  between  .80  and  1 .20.  We  selected  these  values  because  they  indicate  that 
no  greater  than  20%  of  applicants’  choices  (as  a  group)  are  being  incorrectly 
predicted — an  error  rate  comparable  to  those  of  most  prediction  problems. 

A  review  of  Tables  C.l  through  C.4  indicates  the  following.  First,  in  terms  of  predicting 
applicants’  choices  to  join  (or  not)  the  Army,  the  performance  of  the  JCM  is  strong.  Across  all 
subgroups  and  quarters,  differences  between  actual  and  expected  choices  are  small,  as  evident  by 
“Diff(A-E)”  values  uniformly  close  to  zero  and  “Ratio(E/A)”  close  to  1 .00.  With  the  exception 
of  AFQT  Cat  IV  applicants  (for  the  Second  and  Third  Quarters),  for  which  sample  sizes  tend  to 
be  small,  the  error  rates  (see  the  “Ratio(E/A)”  values)  do  not  exceed  4%.  For  applicants  as  a 
whole  (see  Subgroup  labeled  “All”),  the  JCM  is  correctly  predicting  close  to  100%  of  applicants’ 
decisions  to  join  (or  not)  the  Army.  Second,  regarding  applicants’  MOS  opportunity  choices,  the 
diagnostics  indicate  that  the  predictive  efficacy  of  the  JCM  is  good  across  subgroups  and  job 
families.  With  the  exception  of  job  families  with  MOS  that  are  infrequently  offered  (e.g., 
Surveillance  and  Communication-SC)  or  subgroups  with  small  sample  sizes  (AFQT  Cat  IVs) 
and/or  combinations  of  the  two  (e.g..  Female  applicants  for  Field  Artillery-FA  positions), 
“Diff(A-E)”  values  are  consistently  close  to  zero  across  all  four  quarters.  Similarly,  while  there 
is  greater  variability  in  “Ratio(E/A)”  values  than  when  predicting  Army  enlistment  decisions, 
these  values  are  consistently  within  the  accepted  criteria  for  all  subgroups  and  job  families  (.80< 
Ratio(x)  <1 .20).  For  applicants  as  a  whole  (see  Subgroup  labeled  “All”),  these  error  rates  do  not 
exceed  9%,  even  for  the  less  critical  job  families  (e.g.,  SC).  This  means  that  overall,  the  JCM  is 
correctly  predicting  roughly  91%+  of  applicants’  MOS  opportunity  choices  across  the  four 
quarters. 


Out-of-Sample  Prediction 

To  examine  the  predictive  performance  of  the  estimated  JCM  model  we  carried  out  the 
same  diagnostics  used  above  for  model  fit  on  the  out-of-sample  observations  from  each  quarter. 
Predictive  performance  diagnostics  at  the  subgroup  and  job  family  level  are  reported  by  quarter 
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in  Appendix  D  (Tables  D.l  through  D.4).  Note  that  there  are  row  entries  in  these  tables  with 
zeroes  under  column  “Actual(A)”  and  undefined  under  column  “Ratio(E/A).”  These  rows 
correspond  to  relatively  small  subgroup-job  family  combinations  that  were  all  included  in  the 
estimation  sample. 

At  the  overall  Army  level  (subgroup  labeled  “All”),  the  estimated  probabilities  of  an 
applicant  selecting  an  MOS  from  a  given  job  family  are  all  relatively  close  to  the  actual 
percentages,  with  the  exception  of  Surveillance  &  Communications  (SC)  which  accounted  for  a 
very  small  percentage  of  the  total  opportunities.  The  magnitude  of  the  differences  between 
estimated  and  actual  percentage  for  SC  across  all  EIRB  quarters  is  acceptable  given  the  intended 
application  of  the  JCM  and  total  frequency  SC  opportunities  were  offered.  In  terms  of  the 
decision  to  join  or  not  to  join  the  Army,  the  differences  between  the  expected  and  actual 
percentages  are  also  small  for  all  quarters. 

At  the  subgroup  level,  the  predictive  performance  remains  satisfactory  overall  across  all 
quarters.  However,  the  diagnostics  indicate  more  instances  of  over  or  under  predicting  outside  of 
the  desirable  range  (Ratio  (E/A)  between  .80  and  1.20)  even  for  AFQT  non-Cat  IV  applicants. 

An  examination  of  the  estimated  and  actual  percentages,  however,  suggests  that  for  most  of  these 
cases  the  discrepancies  are  relatively  not  too  large  for  our  intended  application. 

In  sum,  the  overall  and  subgroup  diagnostics  indicate  that  the  JCM  model  fit  is  high  and 
that  it  has  good  predictive  accuracy  for  our  application. 
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SIMULATING  ARMY  APPLICANTS’  JOB  CHOICES 


To  implement  the  JCM  in  the  EPAS  Simulation,  we  developed  a  procedure  for  simulating 
a  randomized  job  choice  for  each  applicant.  As  an  overview,  this  job  choice  randomization 
procedure  involves  two  stages.  In  the  first  stage,  percentages  are  computed  that  represent  the 
“attractiveness”  (i.e.,  utilities)  of  the  different  job  opportunities  to  the  applicant.  In  the  second 
stage,  a  random  choice  decision  is  then  generated  for  the  applicant  using  these  attractivenesss 
percentages.  As  discussed  previously,  the  choice  decision  generated  results  in  the  applicant 
either:  (a)  deciding  to  not  join  the  Army;  or  (b)  selecting  one  of  the  available  job  opportunities 
from  their  list. 

The  computational  details  and  data  input  requirements  involved  in  these  two  stages, 
including  step-by-step  instructions  for  implementing  the  procedure  in  the  EPAS  Simulation 
environment,  are  documented  in  the  following  sections.  In  describing  the  procedure,  no 
distinction  is  made  between  REQUEST  and  EPAS-Enhanced  REQUEST  simulation  conditions. 
This  is  because  the  computational  steps  in  procedure  are  independent  of  the  condition  underlying 
the  rank  ordering  of  the  job  opportunities.  To  illustrate  our  procedure  (and  its  implementation), 
an  example  is  provided  in  the  accompanying  MS  Excel  workbook. 

First  Stage:  Computing  Attractiveness  Percentages 

The  objective  in  the  first  stage  is  to  compute  attractiveness  percentages  for  each  MOS- 
reception  date  row  opportunity  in  the  job  list  of  an  applicant.  These  percentages  become  the 
input  for  the  second  stage,  where  the  applicant’s  randomized  choice  deeision  is  generated  to 
reflect  either:  (a)  a  decision  not  to  join  the  Army;  or  (b)  the  selection  of  a  specific  MOS- 
reception  date  (job)  opportunity.  The  algorithm  described  below  is  developed  around  a  JCM 
auxiliary  table.  The  purpose  of  this  table  is  to  store  intermediate  values  in  the  calculation  of  the 
attractiveness  percentages. 

Steps  for  Computing  Attractiveness  Percentages 

Table  5  identifies  the  columns  in  the  JCM  auxiliary  table  relevant  to  computing 
attractiveness  percentages.  The  rows  in  the  auxiliary  table  are  indexed  by:  (1)  applicant’s  SSN 
(IND_SSN);  (2)  date  and  time  search  was  completed  (CONTRACT_DATE);  and  (3)  the  MOS  and 
reception  date  (RECSTA_DT)  of  the  job  opportunity.  In  other  words,  the  row  dimension  exactly 
conforms  to  that  of  the  job  opportunity  data,  with  one  record  for  each  MOS-reception  date 
opportunity  in  the  job  list  of  each  applicant.  Column  ATTRACT_PCT  represents  the  attractiveness 
percentage,  which  is  the  final  output  in  this  stage.  Columns  WGT_ACCESS  through 
EPSCORE_TOT  are  intermediate  values  in  the  computation  of  ATTRACT_PCT.  The  second 
column  in  Table  5  indicates  whether  the  value  of  a  column  in  the  auxiliary  table  will  be  supplied 
(fixed=Y)  or  will  be  computed  during  the  simulation  (fixed=N).  Columns  in  the  auxiliary  table 
that  are  fixed  (fixed=Y)  contain  “pre-computed”  values  based  on  applicant  demographics,  test 
scores,  and  incentives  offered  with  the  MOS  opportunities  in  an  applicant’s  job  list.  Columns 
containing  values  to  be  computed  during  the  simulations  (fixed=N)  depend  on  the  ranking  of 
opportunities  in  the  job  list. 
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Table  5.  JCM  Auxiliary  Table  for  Computing  Job  Attractiveness  Percentage 


Column 

FIXED 

Description 

IND  SSN 

Y 

applicant’s  SSN 

CONTRACT  DATE 

Y 

date/time  search  was  completed 

date/time 

MOS 

Y 

MOS  of  job  opportunity 

RECSTA  DT 

Y 

reception  date  of  job  opportunity 

date 

WGT  ACCESS 

Y 

accession  percentage  weight 

WGT  RANK 

Y 

weight  of  rank  in  preference  score 

RANK  PCT 

N 

rank  of  opportunity  in  percentage 

Y 

known  part  of  preference  score 

number(10,8) 

PSCORE  RANK 

N 

rank  based  part  of  preference  score 

EPSCORE  TOT 

N 

total  preference  score 

ATTRACT  PCT 

N 

attractiveness  percentage 

Note:  Depending  on  how  computations  in  the  first  stage  are  implemented,  columns  PSCORE_RANK  and 
EPSCORE_TOT  in  the  JCM  auxiliary  table  can  be  made  optional 


The  following  steps  describe  the  computations  needed  to  obtain  the  ATTRACT_PCT  values 
for  the  opportunities  in  an  applicant’s  job  list.  In  the  expressions  below,  job  opportunities  for  a 
given  applicant  are  identified  (or  indexed)  using  the  SSN  (/)  of  the  applicant,  and  the  MOS  {m) 
and  reception  date  {d)  of  each  opportunity. 


Step  1.  Compute  RANK_PCT.  Convert  the  integer  rank  order  of  each  row  opportunity  in  the  job 
list  to  its  percentage  equivalent  using  the  following  formula: 


„  I-  ji  ■'*<)('.  "1. 1^) 
. 


where 


Ro{i,m,d)  = 

N{i)  = 
Rp{i,m,d)  = 


Integer  rank  order  of  the  row  opportunity  identified  by  MOS  m  and 
reception  date  d  in  the  job  list  of  the  zth  applicant. 

Total  number  of  opportunities  in  the  job  list  of  the  zth  applicant. 

Percentage  rank  equivalent  of  Rq  {i,m,d) 


Note  that  RQ{i,m,d)  will  be  based  on  the  rank  ordering  of  REQUEST  or  EPAS-Enhanced 
REQUEST  conditions. 


Step  2.  Compute  PSCORE_RANK.  Compute  the  component  of  the  preference  score  that  is 
dependent  on  the  rank  of  the  job  opportunity: 

Sp (z, m,d)  =  \00xWp{i)x  Rp{i,m,d) 


where 
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JVjf  (/)  =  Weight  applied  to  percentage  ranks  of  job  opportunities  of  the  /th 
applicant  (WGT_RANK). 

Sjf  (/,  m,  d)  =  Rank-based  component  of  preference  score  of  the  rth  applicant  for  the 
job  opportunity  identified  by  MOS  m  and  reception  date  d. 


Step  3.  Compute  EPSCORE_TOT.  Compute  the  exponential  of  total  preference  scores  for  each 
row  opportunity  in  the  job  list: 

Ej  (/,  m,d)=^  expl-Sy^  (/,  m,  d)  +  S„{i  ,m,  d)} 


where 

Sj.  (/,  m,  d)  =  FIXED  component  of  preference  score  of  the  /th  applicant  for  the  job 
opportunity  identified  by  MOS  m  and  reception  date  d 
(PSCORE_fixed). 

Ej  (/,  m,d)  -  Exponential  of  total  preference  score  of  the  /th  applicant  for  the  job 
opportunity  identified  by  MOS  m  and  reception  date  d 
(EPSCORE_TOT). 

Note  that  by  using  Ej.  (/,  m,d)  =  expl^^^.  (/,  m,  d)  +  W,^{i)xRp{i,m,d)},  Step  2  can  be  skipped 
and  column  PSCORE_RANK  dropped  from  the  JCM  auxiliary  table. 


Step  4.  Compute  ATTRACT_PCT.  Convert  the  total  preference  scores  of  row  opportunities  into 
attractiveness  percentage  values  using  the  following  formulas: 


(1)  A,{i,m,d)  =  W^{i)x 

Et{i) 

(2)  £7.  (/)=  2]  £■.,.(/, 


where 

E-j  (i)  =  Sum  of  exponential  of  total  preference  scores  E.j{i,m,d)  across  all  row 
opportunities  in  the  job  list  of  the  /th  applicant. 

(/)  =  Accession  percentage  weight  for  the  /th  applicant  (WGT_ACCESS). 

Aj,  (/,  m,  d)  =  Attractiveness  percentage  of  the  /th  applicant  for  the  job  opportunity 
identified  by  MOS  m  and  reception  date  d  (ATTRACT_PCT) 

Note  that  the  attractiveness  percentage  values  A,,{i,m,d)  across  opportunities  in  the  job  list 
of  the  /th  applicants  add  up  to  which  generally  is  less  than  or  equal  to  100  percent 

(see  additional  details  in  the  next  section). 
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Table  6.  Example  JCM  Auxiliary  Table  Rows  for  a  Single  Applicant _ 

IND_SSN  CONTRACT  DATE  MIPS  RECSTA  DT  WGT_ACCESS  WGT_RANK  RANK_PCT  PSCORE_FIXED  PSCORE_  RANK  EPSCORE_TOT  ATTRACT_PCT 

123456789  10/26/01  llx  11/19/01  0.92738761-0.308527380.04000000  4.87461438  -0.01234110129.31784399  0.21795323 
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Stage  One  Example 

The  following  example  illustrates  the  steps  in  the  computation  of  the  attractiveness 
percentages.  Table  6  shows  the  rows  in  the  completed  JCM  auxiliary  table  for  a  single  applicant. 
There  are  a  total  of  25  opportunities  in  the  job  list  for  this  applicant. 

In  Step  1 ,  the  integer  ranks  1  through  25  were  converted  to  their  percentage  equivalent 
(RANK_PCT)  in  decimal  form  by  dividing  each  by  25.  In  Step  2,  each  of  these  rank  percentages 
was  then  multiplied  by  -.30852738,  the  value  of  WGT_RANK,  to  obtain  PSCORE_RANK.  Note 
that  WGT_RANK  can  (and  will)  differ  across  applicants,  but  is  constant  for  the  same  applicant. 

The  computed  value  of  PSCORE_RANK  is  the  component  of  the  preference  score  that  is 
dependent  on  rank. 

Continuing  with  the  example,  in  Step  3,  the  exponential  of  the  sum  of  PSCORE_FIXED 
and  PSCORE_RANK  was  calculated  to  obtain  EPSCORE_TOT  for  each  of  the  25  job  opportunities. 
For  example,  looking  at  the  MOS  1 IX  opportunity,  EPSCORE_TOT  was  evaluated  as 
ex/?(4. 87461438-0. 012341 10),  which  is  equal  to  129.31784399.  In  Step  4,  EPSCORE_TOT 
values  for  all  25  opportunities  of  the  applicant  were  initially  converted  to  percentages  by 
dividing  each  by  their  total  (550.24542601).  To  obtain  ATTRACT_PCT,  each  percentage  was  then 
multiplied  by  WGT_ACCESS  (0.92738761).  Note  that  the  attractiveness  percentages  at  the  end  of 
Step  4  add  up  to  0.92738761,  which  is  equal  to  the  value  of  WGT_ACCESS  (as  it  should). 

Second  Stage;  Generating  An  Applicant’s  Job  Choice 

In  the  second  stage  the  goal  is  to  generate  a  randomized  job  choice.  This  choice  decision 
reflects  an  applicant’s  decision  to  either:  (a)  not  join  the  Army;  or  (2)  select  one  of  the  available 
job  opportunities  from  their  job  list,  using  a  randomization  procedure  that  is  consistent  with  the 
JCM.  By  incorporating  a  randomized  component  in  simulating  applicant  job  choice,  the 
accuracy  of  the  JCM  for  modeling  real-world  applicant  decisions  is  enhanced.  To  ensure  that 
across  multiple  replications  an  applicant’s  simulated  job  choice  generally  corresponds  to  their 
actual  decision,  this  randomization  procedure  must  produce,  on  the  average,  selection 
probabilities  (percentages)  that  are  equal  to  the  attractiveness  percentages  of  opportunities  in  the 
job  list  computed  in  Step  4  of  the  previous  section.  A  chance  algorithm  that  satisfies  this 
requirement  is  described  below. 

Steps  in  Generating  Applicant’s  Randomized  Job  Choice 

To  simplify  the  discussion,  the  applicant  index  /  will  be  dropped  from  our  notation  and 
the  MOS-reception  date  opportunities  will  be  labeled  by  {Oi,  O2,  ...,  Oj,  ...,  Oa/},  where  is  the 
total  number  of  opportunities  in  the  job  list  of  an  applicant.  It  is  not  necessary  for  the  sequence  of 
jobs  in  this  set  to  correspond  to  an  actual  rank  ordering.  Similarly,  we  will  denote  the 
attractiveness  percentages  computed  from  Section  2  hy  A p{j),  fory  =  1 ,  2,. . .,  A,  in  the  same  order 
as  the  job  list  sequence. 

Using  the  preceding  notations,  the  chance  algorithm  for  randomly  assigning  an  applicant 
to  one  of  the  jobs  in  {Oi ,  O2,  ...,  Oj ,  Oy+/ ,  ...,  O^},  given  the  applicant’s  attractiveness 
percentages  {Ap{J),  Ap(2),...,Ap(J),...,Ap{N)},  is  described  by  the  following  steps. 
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Step  1.  Construct  Job  Look-Up  Intervals.  Partition  the  unit  interval  (0,1)  into  N+1  sub-intervals 
or  “bins”: 


where  =  0 ,  =  1  ,  and  the  remaining  lower  and  upper  bounds  are  computed  from  the 

applicant’s  attractiveness  percentages  using: 


,  ,j  =  N  +  \ 


N 


I,  =t/,.,,y  =  2,3,. 


,N  +  \ 


Note  that  must  be  equal  to  \-Wy^. 

Step  2.  Generate  a  Decision  Random  Number.  Generate  a  pseudo  random  number  D,  0  <  D  <  1 . 

Step  2.  Simulate  Applicant's  Choice  Decision.  Using  the  random  number  D,  compute  the  job 
index  value  using  the  rule: 


=  maxz{y|y  =  l,2,...,A^} 

D>Lj 

The  first  expression  states  that  indexes  the  job  associated  with  the  first  upper  bound  that 
is  greater  than  or  equal  to  D  (i.e.,  Uj^  is  the  minimum  among  Uj  s  above  D).  Similarly,  the 
second  expression  states  that  y'o  indexes  the  job  associated  with  the  last  lower  bound  that  is 
less  than  D  (i.e.,  Lj^  is  the  maximum  among  Lj  s  below  D).  When  programming  to 

implement  this  step,  it  may  be  more  convenient  to  use  one  form  over  the  other  depending  on 
the  available  routines. 

The  job  index  value  is  interpreted  in  terms  of  the  applicant  simulated  choice  decision  as 
follows: 


Access:  If  <W  the  applicant  chooses  job  Oj^. 

Not  Access:  If  =  N+1,  the  applicant  decides  not  to  access  or  join  the  Army. 
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Alternatively,  the  applicant’s  simulated  choice  decision  can  also  be  expressed  directly  in  terms  of 
the  random  number  D  as  follows: 

Access:  If  <  D  <  U ,  the  applicant  chooses  job  Oj^  . 

Not  Access:  If  D>U  the  applicant  decides  not  to  access  or  join  the  Army. 


The  following  comments  pertain  to  actual  EPAS  simulation  runs.  Under  pure  REQUEST 
condition,  the  attractiveness  percentages  computed  in  Step  4  of  Section  2  will  remain  constant 
across  replications  of  the  simulation;  that  is,  Ap{i,m,d)  is  constant  for  specified  values  of  i,  m, 
and  d.  However,  under  EPAS-Enhance  REQUEST  condition,  attractiveness  percentages  for  a 
given  applicant  can  vary  across  replications.  These  percentages  will  depend  on  EPAS  rank 
ordering  of  opportunities  for  the  week  (or  EPAS  optimization  period)  of  the  search  date,  which 
itself  is  a  function  of  all  simulated  choice  decisions  during  the  period  preceding  the  search  week 
of  an  applicant. 

Stage  Two  Example 

The  following  discussion  continues  the  example  introduced  in  the  preceding  section.  At 
the  end  of  that  discussion,  attractiveness  percentages  were  obtained  for  each  opportunity  in  the 
job  list  of  the  applicant.  The  25  opportunities  in  the  list  are  again  shown  in  Table  7,  along  with 
two  columns  for  the  lower  and  upper  bounds  of  the  26  look-up  intervals.  The  26th  interval 
corresponds  to  the  choice  decision  not  to  join  the  Army  (not  access),  which  is  represented  by  the 
value  000  at  the  bottom  of  the  MOS  column. 

The  26  look-up  intervals  defined  by  the  last  two  columns  in  Table  7  were  constructed 
using  the  formulas  in  Step  1 .  First,  the  upper  bound  of  the  interval  for  the  first  opportunity  in  the 
job  sequence  (MOS  1 IX)  was  set  to  0.21795323,  the  attractiveness  percentage  of  the  first 
opportunity  obtained  from  Table  6.  Then  the  upper  bounds  of  the  remaining  opportunities  were 
computed  by  adding  their  respective  attractiveness  percentages  to  the  upper  bound  of  the 
preceding  opportunity  in  the  job  sequence.  For  example,  attractiveness  of  the  second 
opportunity  is  0.02431 167,  thus  its  upper  bound  is  equal  to  0.02431 167+0.21795323  or 
0.24226490.  The  lower  bound  for  the  first  opportunity  was  set  to  zero,  while  lower  bounds  for 
all  other  opportunities  were  set  respectively  to  the  upper  bounds  of  the  preceding  opportunities  in 
the  job  sequence. 

After  constructing  the  look-up  table  above,  job  choices  can  be  simulated  for  the  applicant 
using  Steps  2  and  3  of  the  second  stage.  This  is  illustrated  as  follows  using  the  lower  bound 
version  of  Step  3.  Suppose  the  decision  random  number  D  generated  in  Step  2  is  equal  to 
0.36596490.  Going  down  column  LOWER_BND  in  Table  7,  the  maximum  lower  bound  that  is 
less  than  0.36596490  was  0.33271687,  which  corresponds  to  the  fifth  job  in  the  list  (MOS  14  R). 
This  would  be  the  simulated  job  choice  of  the  applicant  for  the  given  realization  of  the  random 
number  D. 
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Table  7.  Example  Job  Look-Up  Intervals  for  Generating  Choice  Decisions 


JOB  IDX  IND  SSN 


1  123456789 

2  123456789 

3  123456789 
123456789 
123456789 
123456789 
123456789 
123456789 
123456789 
123456789 
123456789 
123456789 
123456789 
123456789 
123456789 
123456789 
123456789 
123456789 
123456789 
123456789 
123456789 
123456789 
123456789 
123456789 
123456789 
123456789 


CONTRACT  DATE  MOS  RECSTA  DT  LOWER  BND  UPPER  BN 


10/26/01  11 

10/26/01  13 

10/26/01  14 

10/26/01  14 

10/26/01  14 

10/26/01  14 

10/26/01  19 

10/26/01  93 

10/26/01  31 

10/26/01  63 

10/26/01  6“ 

10/26/01 
10/26/01 
10/26/01 
10/26/01 
10/26/01 
10/26/01 
10/26/01 
10/26/01 
10/26/01 
10/26/01 
10/26/01 
10/26/01 
10/26/01 
10/26/01 
10/26/01 


LX  1 

1/1 

■ 

3M  1 

1/0 

■ 

JE  1 

1/1 

2/0 

1  0 

IJ  1 

1/2 

6/0 

1  0 

IR  1 

1/1 

9/0 

1  0 

IT  1 

1/2 

1  0 

3D  1 

1/0 

1  0 

iC  1 

1/2 

6/0 

1  0 

LR  1 

1/2 

6/0 

1  0 

3A  1 

1/1 

9/0 

1  0 

?M  1 

5X  1 

IS  1 

1/0 

1/2 

1/2 

5M  1 

1/1 

1  0 

ID  1 

1/1 

?B  1 

1/2 

6/0 

1  0 

W  1 

1/0 

5/0 

1  0 

3G  1 

1/0 

mi 

3H  1 

1/2 

6/0 

1  0 

LS  1 

1/1 

2/0 

1  0 

LW  1 

1/2 

6/0 

1  0 

jB  1 

1/0 

5/0 

1  0 

jy  1 

1/1 

2/0 

1  0 

)G  1 

1/1 

2/0 

1  0 

3J  1 

1/1 

9/0 

1  0 

)0 

0 

Alternative  MOS  choices  are  shown  in  Table  8  for  different  realizations  of  the  decision 
random  number  D  using  the  look-up  intervals  in  Table  7.  Since  Table  7  was  fixed  here,  this 
section  of  the  example  can  only  apply  to  simulation  replications  for  REQUEST  condition.  Note 
that  in  simulation  replication  number  17,  the  applicant  “chooses”  not  to  join  the  Army  —  as  with 
the  REQUEST  transaction  data,  the  applicant  would  not  have  a  reservation  for  this  replication. 


Table  8.  Example  Job  Choice  Decisions  for  Realizations  of  D 


SIM_REP 

D_  RAND 

CHOICE 

SIM.  REP 

D_RAND 

CHOICE 

SIM  REP 

D  RAND 

CHOICE 

1 

0.36596490 

14R 

11 

0.73552556 

68G 

11 

0.67483575 

35M 

2 

0.25609965 

14E 

12 

0.23393994 

13M 

22 

0.11551405 

IIX 

3 

0.01446337 

IIX 

13 

0.62987706 

13X 

23 

0.54983242 

31R 

4 

0.24730711 

14E 

14 

0.41649715 

19D 

24 

0.65373572 

14S 

5 

0.01710986 

IIX 

15 

0.58733379 

63A 

25 

0.25938727 

14E 

6 

0.59452253 

63A 

16 

0.21763324 

IIX 

26 

0.82558274 

91W 

7 

0.42728675 

19D 

17 

0.92913210 

000 

27 

0.37955547 

14T 

8 

0.90603971 

35Y 

18 

0.04151490 

IIX 

28 

0.53611325 

31R 

9 

0.37711992 

14T 

19 

0.02839122 

lix 

29 

0.69975815 

62B 

10 

0.80787759 

91W 

20 

0.77322239 

91W 

30 

0.06908547 

IIX 
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Example  Excel  Workbook 

The  accompanying  Excel  workbook  contains  the  data  and  tables  used  in  the  examples 
presented  in  this  document  to  illustrate  the  job  choice  randomization  procedure.  There  are  a  total 
of  four  worksheets  in  the  workbook.  The  following  is  a  brief  description  of  each: 

•  OppDATA.  This  worksheet  contains  25  job  opportunities  for  a  single  applicant  (only 
columns  used  in  the  procedure  are  shown).  The  column  RANK  can  represent  the  job 
ordering  based  on  the  REQUEST  or  EPAS-Enhanced  REQUEST  conditions.  Note  that 
rows  need  not  be  sorted  by  RANK. 

•  AuxTBL.  This  worksheet  represents  the  JCM  auxiliary  table  and  calculations  described  in 
the  “First  Stage:  Computing  Attractiveness  Percentages”  section  (see  Tables  5  and  6). 
Columns  in  black  font  correspond  to  supplied  values,  while  columns  in  blue  font  were 
computed  in  the  worksheet  using  Excel  functions/formulas. 

•  LookUpTBL.  This  worksheet  is  an  example  of  the  look-up  table  and  chance  algorithm 
described  in  the  “Second  Stage:  Generating  An  Applicant’s  Job  Choice”  section  (see 
Table  7).  For  this  particular  example,  the  lower  bound  version  of  the  rule  given  in  Step  3 
of  the  algorithm  was  implemented. 

•  SimTBL.  The  rows  in  this  worksheet  show  the  result  of  30  separate  and  independent 
simulation  replications  for  the  single  applicant  (see  Table  8).  Values  under  D_RAND  were 
generated  using  Excel  rand()  function.  As  a  reminder,  the  special  MOS  value  equal  to  000 
indicates  that  the  applicant  decided  to  not  join  the  Army. 
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SIMULATION  RESULTS 


IND_SSN 

123456789 

123456789 

123456789 

123456789 

123456789 

123456789 

123456789 

123456789 

123456789 

123456789 

123456789 

123456789 

123456789 

123456789 

123456789 

123456789 

123456789 

123456789 

123456789 

123456789 

123456789 

123456789 

123456789 

123456789 

123456789 

123456789 

123456789 

123456789 

123456789 

123456789 


CONTRACT_DATE 

SIM_REP 

D_RAND 

CHOICE 

10/2672001 

1 

0.36596490 

14R 

10/26/2001 

2 

0.25609965 

14E 

10/26/2001 

3 

0.01446337 

11X 

10/26/2001 

4 

0.24730711 

14E 

10/26/2001 

5 

0.01710986 

11X 

10/26/2001 

6 

0.59452253 

63A 

10/26/2001 

7 

0.42728675 

19D 

10/26/2001 

8 

0.90603971 

35Y 

10/26/2001 

9 

0.37711992 

14T 

10/26/2001 

10 

0.80787759 

91W 

10/26/2001 

11 

0.73552556 

68G 

10/26/2001 

12 

0.23393994 

13M 

10/26/2001 

13 

0.62987706 

13X 

10/26/2001 

14 

0.41649715 

19D 

10/26/2001 

15 

0.58733379 

63A 

10/26/2001 

16 

0.21763324 

11X 

10/26/2001 

17 

0.92913210 

000 

10/26/2001 

18 

0.04151490 

11X 

10/26/2001 

19 

0.02839122 

11X 

10/26/2001 

20 

0.77322239 

91W 

10/26/2001 

21 

0.67483575 

35M 

10/26/2001 

22 

0.11551405 

11X 

10/26/2001 

23 

0.54983242 

31R 

10/26/2001 

24 

0.65373572 

14S 

10/26/2001 

25 

0.25938727 

14E 

10/26/2001 

26 

0.82558274 

91W 

10/26/2001 

27 

0.37955547 

14T 

10/26/2001 

28 

0.53611325 

31R 

10/26/2001 

29 

0.69975815 

62  B 

10/26/2001 

30 

0.06908547 

11X 
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AUXILIARY  TABLE 


blue  font  blue  font  blue  font  blue  font 


IND_SSN 

CONTRACT_DATE 

MOS 

RECSTA^DT 

WGT^ACCESS 

WGT^RANK 

RANK.PCT 

PSC0RE_F1XED 

PSCORE_  RANK 

EPSCORE_TOT 

A'rrRACT_PCT 

123456789 

10/26/2001 

11X 

11/1  97oi 

0.92738761 

-0.30852738 

0.04000000 

4.87461438 

-0.01234110 

129.31784399 

0.21795323 

123456789 

10/26/2001 

13M 

11/05/01 

0.92738761 

-0.30852738 

0.08000000 

2.69363153 

-0.02468219 

14.42480561 

0.02431167 

123456789 

10/26/2001 

14E 

11/12/01 

0.92738761 

-0.30852738 

0.12000000 

3.43228909 

-0.03702329 

29.82257933 

0.05026319 

123456789 

10/26/2001 

14J 

11/26/01 

0.92738761 

-0.30852738 

0.16000000 

3.22094504 

-0.04936438 

23.84514558 

0.04018878 

123456789 

10/26/2001 

14R 

11/19/01 

0.92738761 

-0.30852738 

0.20000000 

3.20961414 

-0.06170548 

23.28731197 

0.03924860 

123456789 

10/26/2001 

14T 

11/26/01 

0.92738761 

-0.30852738 

0.24000000 

2.96489904 

-0.07404657 

18,00865484 

0.03035192 

123456789 

10/26/2001 

19D 

11/05/01 

0,92738761 

-0.30852738 

0.28000000 

3.85587978 

-0.08638767 

43.35803811 

0.07307595 

123456789 

10/26/2001 

93C 

11/26/01 

0.92738761 

-0.30852738 

0.32000000 

2.62821798 

-0.09872876 

12.54709565 

0.02114697 

^23456789 

10/26/2001 

31R 

11/26/01 

0.92738761 

-0.30852738 

0.36000000 

3.96649586 

-0.11106986 

47.24874065 

0.07963337 

123456789 

10/26/2001 

63A 

11/19/01 

0.92738761 

-0.30852738 

0.40000000 

2.96425564 

-0.12341095 

17.13022916 

0.02887141 

123456789 

10/26/2001 

63M 

11/05/01 

0.92738761 

-0.30852738 

0.44000000 

2.53579504 

-0.13575205 

11.02365033 

0.01857934 

123456789 

10/26/2001 

13X 

11/26/01 

0.92738761 

-0.30852738 

0.48000000 

2.67137423 

-0.14809314 

12.46944291 

0.02101609 

123456789 

10/26/2001 

14S 

11/26/01 

0.92738761 

-0.30852738 

0.52000000 

2.31764614 

-0.16043424 

8.64699533 

0.01457371 

123456789 

10/26/2001 

35M 

11/12/01 

0.92738761 

-0.30852738 

0.56000000 

2.94551118 

-0.17277533 

16.00235415 

0.02697048 

123456789 

10/26/2001 

52D 

11/19/01 

0.92738761 

-0.30852738 

0.60000000 

1.90261674 

-0.18511643 

5.57058633 

0.00938871 

123456789 

10/26/2001 

62B 

11/26/01 

0.92738761 

-0.30852738 

0.64000000 

1.23870464 

-0.19745752 

2.83274759 

0.00477433 

123456789 

10/26/2001 

67U 

11/05/01 

0,92738761 

-0.30852738 

0.68000000 

3.20701730 

-0.20979862 

20.02975031 

0.03375829 

123456789 

10/26/2001 

68G 

11/05/01 

0.92738761 

-0.30852738 

0.72000000 

1.66273864 

-0.22213971 

4.22322447 

0.00711785 

123456789 

10/26/2001 

68  H 

11/26/01 

0.92738761 

-0.30852738 

0.76000000 

1.61509864 

-0.23448081 

3.97735821 

0.00670347 

123456789 

10/26/2001 

68S 

11/12/01 

0.92738761 

-0.30852738 

0.80000000 

2.72246134 

-0.24682190 

11.88930716 

0.02003832 

123456789 

10/26/2001 

91W 

11/26/01 

0.92738761 

-0.30852738 

0.84000000 

4.55841269 

-0.25916300 

73.64451667 

0.12412100 

123456789 

10/26/2001 

55B 

11/05/01 

0,92738761 

-0.30852738 

0.88000000 

2.07597963 

-0.27150409 

6.07678357 

0.01024185 

123456789 

10/26/2001 

35Y 

11/12/01 

0.92738761 

-0.30852738 

0.92000000 

2.77416449 

-0.28384519 

12.06512789 

0.02033465 

123456789 

10/26/2001 

63G 

11/12/01 

0.92738761 

-0.30852738 

0.96000000 

1.17920624 

-0.29618628 

2.41819153 

0.00407564 

123456789 

10/26/2001 

63J 

11/19/01 

0.92738761 

-0.30852738 

1.00000000 

-0,64612826 

-0.30852738 

0.38494468 

0.00064879 

SUM_EPSC0RE_T0T=  550.24542601 
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LOOK-UP  TABLE 


IND_SSN 

CONTRACT_DATE 

MOS 

RECSTA_DT 

JOBJDX 

LOWER_BND 

UPPER_BND 

123456789 

10/26/2001 

11X 

11/19/2001 

1 

0.00000000 

0.21795323 

123456789 

10/26/2001 

13M 

11/05/2001 

2 

0.21795323 

0.24226490 

123456789 

10/26/2001 

14E 

11/12/2001 

3 

0.24226490 

0.29252809 

123456789 

10/26/2001 

14J 

11/26/2001 

4 

0.29252809 

0.33271687 

123456789 

10/26/2001 

14R 

11/19/2001 

5 

0.33271687 

0.37196547 

123456789 

10/26/2001 

14T 

11/26/2001 

6 

0.37196547 

0.40231739 

123456789 

10/26/2001 

19D 

11/05/2001 

7 

0.40231739 

0.47539334 

123456789 

10/26/2001 

93C 

11/26/2001 

8 

0.47539334 

0.49654031 

123456789 

10/26/2001 

31R 

11/26/2001 

9 

0.49654031 

0.57617367 

123456789 

10/26/2001 

63A 

11/19/2001 

10 

0.57617367 

0.60504509 

123456789 

10/26/2001 

63M 

11/05/2001 

11 

0.60504509 

0.62362443 

123456789 

10/26/2001 

13X 

11/26/2001 

12 

0.62362443 

0.64464051 

123456789 

10/26/2001 

14S 

11/26/2001 

13 

0.64464051 

0.65921422 

123456789 

10/26/2001 

35M 

11/12/2001 

14 

0.65921422 

0.68618471 

123456789 

10/26/2001 

52  D 

11/19/2001 

15 

0.68618471 

0.69557341 

123456789 

10/26/2001 

62  B 

11/26/2001 

16 

0.69557341 

0.70034775 

123456789 

10/26/2001 

67U 

11/05/2001 

17 

0.70034775 

0.73410603 

123456789 

10/26/2001 

68G 

11/05/2001 

18 

0.73410603 

0.74122388 

123456789 

10/26/2001 

68H 

11/26/2001 

19 

0.74122388 

0.74792735 

123456789 

10/26/2001 

68S 

11/12/2001 

20 

0.74792735 

0.76796568 

123456789 

10/26/2001 

91W 

11/26/2001 

21 

0.76796568 

0.89208668 

123456789 

10/26/2001 

55B 

11/05/2001 

22 

0.89208668 

0.90232853 

123456789 

10/26/2001 

35Y 

11/12/2001 

23 

0.90232853 

0.92266318 

123456789 

10/26/2001 

63G 

11/12/2001 

24 

0.92266318 

0.92673882 

123456789 

10/26/2001 

63J 

11/19/2001 

25 

0.92673882 

0.92738761 

123456789 

10/26/2001 

000 

26 

0.92738761 

1.00000000 
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SIMULATION  RESULTS 


IND_SSN 

CONTRACT_DATE 

SIM_REP 

D_FIAND 

CHOICE 

123456789 

10/26/2001 

1 

0.36596490 

14R 

123456789 

10/26/2001 

2 

0.25609965 

14E 

123456789 

10/26/2001 

3 

0.01446337 

11X 

123456789 

10/26/2001 

4 

0.24730711 

14E 

123456789 

10/26/2001 

5 

0.01710986 

11X 

123456789 

10/26/2001 

6 

0.59452253 

63A 

123456789 

10/26/2001 

7 

0.42728675 

19D 

123456789 

10/26/2001 

8 

0.90603971 

35Y 

123456789 

10/26/2001 

9 

0.37711992 

14T 

123456789 

10/26/2001 

10 

0.80787759 

91W 

123456789 

10/26/2001 

11 

0.73552556 

68G 

123456789 

10/26/2001 

12 

0.23393994 

13M 

123456789 

10/26/2001 

13 

0.62987706 

13X 

123456789 

10/26/2001 

14 

0.41649715 

19D 

123456789 

10/26/2001 

15 

0.58733379 

63A 

123456789 

10/26/2001 

16 

0.21763324 

11X 

123456789 

10/26/2001 

17 

0.92913210 

000 

123456789 

10/26/2001 

18 

0.04151490 

11X 

123456789 

10/26/2001 

19 

0.02839122 

11X 

123456789 

10/26/2001 

20 

0.77322239 

91W 

123456789 

10/26/2001 

21 

0.67483575 

35M 

123456789 

10/26/2001 

22 

0.11551405 

11X 

123456789 

10/26/2001 

23 

0.54983242 

31R 

123456789 

10/26/2001 

24 

0.65373572 

14S 

123456789 

10/26/2001 

25 

0.25938727 

14E 

123456789 

10/26/2001 

26 

0.82558274 

91W 

123456789 

10/26/2001 

27 

0.37955547 

14T 

123456789 

10/26/2001 

28 

0.53611325 

31R 

123456789 

10/26/2001 

29 

0.69975815 

62  B 

123456789 

10/26/2001 

30 

0.06908547 

11X 
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APPENDIX  A:  MOS  ALTERNATIVES 


A-1 


Table  A.l.  JCM  Alternative  MOS  Configuration 


Alternative 

MOS 

Alternative 

1  MOS 

1 

IIX 

54 

77F 

2 

12B 

55 

82C 

3 

12C 

56 

88H 

4 

13B 

57 

88M 

5 

13F 

58 

i  88N 

6 

13M 

59 

91K 

7 

13? 

60 

91W 

8 

13R 

61 

92A 

9 

13X 

62 

92G 

10 

14E 

63 

92M 

14J 

64 

92R 

12 

14R 

65 

92S 

13 

14S 

66 

92Y 

14 

14T 

67 

93C 

15 

18X 

68 

93P 

16 

19D 

69 

95B 

17 

19K 

70 

95C 

18 

27D 

71 

96B 

19 

31C 

72 

96H 

20 

31F 

73 

^98C 

21 

31L 

74 

98H 

22 

31P 

75 

98J 

23 

31R 

76 

98K 

24 

31S 

77 

98X 

25 

31U 

CO 

(ADI)  73C,73D 

26 

33W 

79 

(AD2)  75B,75F 

27  ; 

35E 

80 

(AMI)  67D, 67S, 68F, 68N, 

28 

51B 

:  68X  i 

29  ; 

52C 

81 

(AM2)  68B,68D,68H 

30  ; 

52D 

82 

(AM3)  68S,68Y 

31 

54B 

83 

(ELI)  27E,35D 

32 

55B 

84 

(EL2)  35F, 35J, 35L,  39B 

33  ; 

55D 

85 

(EL3)  27M,27X 

34  ; 

5  6M 

86  1 

(EL4)  27T, 35H, 35R, 35Y 

35 

62B 

87 

(ELS)  35M,35N 

36  i 

62E 

88 

(FAl)  13C,  13D,  13E,  93F 

37  : 

63A 

89 

(GEl)  51K, 51M, SIR, 51T 

38  : 

63B 

90 

(GE2)  62F,62H,62J 

39  i 

63G 

91 

(MDl)  91C,  91D, 91E, 91G, 

40  ! 

63H 

91H, 91J, 91M, 91P, 

41  1 

63J 

91T,  91V, 91X 

42  : 

63M 

92  ! 

(MD2)  91Q,91R,91S 

43  1 

63S 

93  i 

(Mil)  96U,97B 

44  : 

63W 

94 

(MI2)  96D,96R,97E 

45  ; 

63Y 

95  1 

(MMl)  44B,44E 

46 

67R 

96  j 

(MM2)  45B,45G,45K 

47  1 

67T 

97  j 

(MM3)  45D,63D 

48 

67U 

98  ! 

(PAD  37F,46Q,46R 

49 

68G 

99  i 

(PWl)  77L,77W 

50  1 

71L 

100  i 

(TEl)  81L,81T,82D 

51  ; 

74B 

101  1 

(TSl)  88K,88L 

52  i 

74C 

102  i 

(VSl)  25M,25R,25V 

53  ^ 

75H 

999  1 

(000)  Non-Accession  | 
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Table  B.l.  Alternative-Specific  Constant  Parameter  Estimates  by  Quarter 


Alternative 

First  Quarter 

Second  Quarter 

Third  Quarter 

Fourth  Quarter 

ID 

MOS 

Estimate 

T-stat 

Estimate 

T-stat 

Estimate 

T-stat 

Estimate 

T-stat 

1 

IIX 

0.000000 

FIXED 

0.000000 

FIXED 

0.000000 

FIXED 

0.000000 

FIXED 

2 

12B 

-0.140107 

-2.88 

-0.146416 

-3.67 

-0.172604 

-3.93 

-0.765366 

-4.54 

3 

12C 

-0.495100 

-3.19 

-0.576711 

-2.82 

-0.659907 

-4.86 

-2.211224 

-4.72 

4 

13B 

-0.206888 

-3.32 

-0.314669 

-1.99 

-0.181339 

-3.77 

-1.139220 

-2.86 

5 

13F 

-0.286691 

-3.27 

-0.437152 

-2.35 

-0.283593 

-4.26 

-1.606045 

-3.45 

6 

13M 

-0.379774 

-3.32 

-0.438790 

-2.35 

-0.382982 

-4.48 

-2.212466 

-3.98 

7 

13P 

-0.415970 

-3.32 

-0.485298 

-2.45 

-0.417103 

-4.38 

-2.532765 

-4.16 

8 

13R 

-0.525436 

-3.29 

-0.578549 

-2.63 

-0.514079 

-4.65 

-2 .418278 

-4.11 

9 

13X 

-0.304714 

-3.27 

-0.387317 

-2.21 

-0.295414 

-4.28 

-1 . 314658 

-2.56 

10 

14E 

-0.470440 

-3.28 

-0.550717 

-2.59 

-0.381872 

-4.36 

-1.754094 

-3.45 

11 

14  J 

-0.379180 

-3.29 

-0.545757 

-2.58 

-0.387999 

-4.45 

-1.617506 

-3.34 

12 

14R 

-0.450420 

-3.34 

-0.516706 

-2.52 

-0.330792 

-4.42 

-2.158464 

-3.83 

13 

14S 

-0.365222 

-3.25 

-0.465664 

-2.40 

-0.359830 

-4.39 

-1 . 900696 

-3.54 

14 

14T 

-0.416703 

-3.33 

-0.528000 

-2.55 

-0.448834 

-4.59 

-2.086147 

-3.80 

15 

18X 

NA 

NA 

0.063759 

1.45 

0.401592 

4.03 

1.705427 

4.04 

16 

19D 

-0.202543 

-3.27 

-0.166931 

-3.69 

-0.190319 

-4.12 

-0.694489 

-4.47 

17 

19K 

-0.219788 

-3.30 

-0.147016 

-3.65 

-0.183758 

-4.24 

-0.821023 

-4.85 

18 

27D 

-0.017913 

-0.26 

-0.158959 

-1.14 

0.097489 

1.13 

-0.854650 

-1.91 

19 

31C 

-0.124552 

-1 . 51 

-0.305032 

-1.88 

-0.251683 

-2.67 

-1.135868 

-2.63 

20 

31F 

-0.155712 

-2.26 

-0.313009 

-1.94 

-0.080472 

-1  . 17 

-0.932595 

-2.24 

21 

31L 

-0.334156 

-3.05 

-0.435286 

-2.31 

-0.371854 

-3.83 

-1.921794 

-3.68 

22 

31P 

-0.228902 

-2.60 

-0.371192 

-2.11 

0.014423 

0.17 

-0.779893 

-1.87 

23 

31R 

-0.201327 

-2.63 

-0.337383 

-2.03 

-0.203803 

-2.87 

-1 .112408 

-2.66 

24 

31S 

-0.084691 

-1.41 

-0.259334 

-1.70 

-0.060452 

-0.89 

-0.312503 

-0.79 

25 

31U 

-0.181991 

-2.41 

-0.322437 

-1.97 

-0.194566 

-2.87 

-1.126257 

-2.63 

26 

33W 

-0.194899 

-2.39 

-0.342980 

-2.01 

-0.230478 

-2.73 

-1.102849 

-2.52 

27 

35E 

-0.255244 

-2.73 

-0.417017 

-2.25 

-0.244625 

-2.93 

-1.368707 

-2.95 

28 

51B 

-0.450032 

-3.08 

-0.550751 

-2.54 

-0.609410 

-4.26 

-1 . 694098 

-3.35 

29 

52C 

-0.511439 

-3.26 

-0.674564 

-2.79 

-0.427169 

-4.20 

-2.406377 

-3.98 

30 

52D 

-0.497865 

-3.29 

-0.627608 

-2.72 

-0.365494 

-4.11 

-2.153444 

-3.85 

31 

54B 

-0.331503 

-3.27 

-0.387764 

-2.21 

-0.161032 

-2.73 

-1.528399 

-3.38 

32 

55B 

-0.319738 

-2.77 

-0.485556 

-2.31 

-0.662998 

-3.92 

-2.337218 

-3.77 

33 

55D 

-0.389732 

-2 . 98 

-0.535481 

-2.43 

-0.751067 

-4.34 

-2.787473 

-4.17 

34 

56m 

-0.158115 

-2.08 

-0.268967 

-1.71 

0.341917 

2.91 

-1.441178 

-2.98 

35 

62B 

-0.575859 

-3.31 

-0.670085 

-2.78 

-0.513232 

-4 . 41 

-2.486239 

-3.93 

36 

62E 

-0.345543 

-2.96 

-0.527712 

-2.47 

-0.639442 

-4.20 

-2 . 395016 

-3.92 

37 

63A 

-0.454120 

-3.26 

-0.633075 

-2.72 

-0 .485042 

-4 . 40 

-2 . 329227 

-3.94 

38 

63B 

-0.363260 

-3.23 

-0.446747 

-2.37 

-0.222184 

-3.52 

-1 .434051 

-3.16 

39 

63G 

-0.639343 

-3.27 

-0.682207 

-2.80 

-0.535016 

-4.43 

-2 .469840 

-4.03 

40 

63H 

-0.584212 

-3.34 

-0.655828 

-2.77 

-0.500972 

-4.55 

-2 . 223479 

-3.87 

41 

63J 

-0.681050 

-3.31 

-0.690747 

-2.80 

-0.557423 

-4 . 41 

-2.703770 

-4.14 

42 

63M 

-0.516565 

-3.25 

-0.646712 

-2.75 

-0.474416 

-4.46 

-2.519895 

-4.03 

43 

63S 

-0.464980 

-3.23 

-0.489874 

-2.45 

-0.385812 

-4.10 

-1.973796 

-3.69 

44 

63W 

-0.406710 

-3.22 

-0.536959 

-2.56 

-0.370523 

-4.07 

-1.895563 

-3.67 

45 

63Y 

-0.565686 

-3.32 

-0.665395 

-2.78 

-0.488271 

-4.58 

-2.237992 

-3.89 

46 

67R 

-0.294693 

-2 . 98 

-0.502330 

-2.48 

-0.333218 

-3.65 

-0.842526 

-1.98 

47 

67T 

-0.251185 

-2.80 

-0.410374 

-2.26 

-0.146066 

-2.28 

-0.892822 

-2.10 

48 

67U 

-0.395788 

-3.14 

-0.561817 

-2.60 

-0.308608 

-3.59 

-1 . 621313 

-3.27 

49 

6BG 

-0.557226 

-3.26 

-0.639312 

-2.73 

-0.445261 

-4.16 

-2.261307 

-3.86 

50 

71L 

-0.059541 

-1 .19 

-0.171022 

-1.28 

-0.032876 

-0.55 

-0.809398 

-2.12 
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Table  B.l.  Alternative-Specific  Constant  Parameter  Estimates  by  Quarter  (continued) 


Alternative 

First  Quarter 

Second  Quarter 

Third  Quarter 

Fourth  Quarter 

ID 

MOS 

Estimate 

T-stat 

Estimate 

T-stat 

Estimate 

T-stat 

Estimate 

T-stat 

51 

74B 

0.147538 

2.29 

-0.076685 

-0.64 

.  0.178384 

2.00 

-0.287401 

-0.80 

52 

74C 

-0.210514 

-2.44 

-0.359503 

-2.09 

-0.203380 

-2.88 

-1.283282 

-2.87 

53 

75H 

-0.061854 

-1.09 

-0.222583 

-1.53 

0.005808 

0.08 

-0.948316 

-2.32 

54 

77F 

-0.261181 

-3.28 

-0.361235 

-2.15 

-0.135329 

-3.55 

-1.422208 

-3.33 

55 

82C 

-0.534063 

-3.27 

-0.540546 

-2.54 

-0.508387 

-4.59 

-2.356341 

-4.03 

56 

88H 

-0.396827 

-3.24 

-0.472366 

-2.42 

-0.329119 

-3.93 

-1.543074 

-3.28 

57 

88M 

-0.289764 

-3.17 

-0.328297 

-2.01 

-0.199772 

-3.52 

-1.029922 

-2.56 

58 

88N 

-0.092068 

-1.33 

-0.408341 

-2.13 

-0.154784 

-1.14 

-1.336923 

-2.88 

59 

91K 

-0.121697 

-2.33 

-0.162846 

-1.25 

0.085473 

1.14 

-0.465224 

-1.20 

60 

91W 

-0.158146 

-2.63 

-0.155347 

-1.22 

-0.007673 

-0.16 

-0.553832 

-1.58 

61 

92A 

-0.047844 

-0.86 

-0.220029 

-1.56 

-0.070299 

-1.34 

-0.822696 

-2.16 

62 

92G 

-0.295858 

-3.32 

-0.392124 

-2.25 

-0.263738 

-4.43 

-1 .493208 

-3.38 

63 

92M 

-0.388235 

-3.07 

-0.366079 

-1.98 

-0.242077 

-1.70 

-1.892917 

-3.58 

64 

92R 

-0.342194 

-3.29 

-0.502654 

-2.50 

-0.322158 

-4.30 

-1.885122 

-3.71 

65 

92S 

-0.396295 

-3.13 

-0.428094 

-2.28 

-0.426375 

-3.78 

-2.193055 

-3.84 

66 

92Y 

-0.033240 

-0.79 

-0.198565 

-1.46 

-0.123564 

-2.55 

-0.897698 

-2.33 

67 

93C 

-0.464959 

-3.32 

-0.558644 

-2.61 

-0.469889 

-4.33 

-1.744156 

-3.55 

68 

93P 

-0.328471 

-3.00 

-0.268377 

-1.73 

-0.151062 

-1.75 

-1.212605 

-2.76 

69 

95B 

-0.206871 

-3.01 

-0.279660 

-1.85 

-0.086432 

-2.01 

-0.522400 

-1.54 

70 

95C 

-0.597194 

-3.30 

-0.629843 

-2.73 

-0.482948 

-4.42 

-2.108626 

-3.91 

71 

96B 

0.115599 

1.77 

-0.157022 

-1.18 

0.110666 

1.40 

-0.390047 

-1 . 04 

72 

96H 

-0.285927 

-3.09 

-0.388852 

-2.20 

-0.287905 

-3.07 

-1.541499 

-3.18 

73 

98C 

-0.316448 

-3.11 

-0.418358 

-2.29 

-0.243895 

-3.22 

-1.551696 

-3.40 

74 

98H 

-0.485811 

-3.27 

-0.541645 

-2.56 

-0.516668 

-4.52 

-2.338416 

-4.09 

75 

98J 

-0.293389 

-3.09 

-0.358325 

-2.10 

-0.132691 

-1.95 

-1.564528 

-3.15 

76 

98K 

-0.354209 

-3.02 

-0.531642 

-2.53 

-0.316415 

-3.56 

-2.024493 

-3.79 

77 

98X 

0.169918 

2.32 

0.155872 

1.58 

0.381557 

3.80 

0.922686 

2.19 

78 

ADI 

-0.093330 

-1.12 

-0.127544 

-0.82 

-0.108693 

-1.31 

-0.847271 

-2.07 

79 

AD2 

-0.100154 

-1.49 

-0.154140 

-1.16 

-0.046697 

-0.64 

-0.957133 

-2.38 

80 

AMI 

-0.405439 

-3.13 

-0.526614 

-2.52 

-0.327735 

-3.64 

-1.352214 

-2.91 

81 

AM2 

-0.457977 

-3.21 

-0.614123 

-2.69 

-0.333059 

-3.72 

-1.821596 

-3.48 

82 

AM3 

-0.447347 

-3.19 

-0.575207 

-2.62 

-0.286301 

-3.30 

-1.604989 

-3.24 

83 

ELI 

-0.462445 

-3.14 

-0.586655 

-2.63 

-0.569539 

-4 . 40 

-1.994104 

-3.66 

84 

EL2 

-0.331652 

-2 . 92 

-0.428985 

-2.28 

-0.247439 

-2.89 

-1.240077 

-2.72 

85 

EL3 

-0.547072 

-3.17 

-0.653817 

-2.74 

-0.607658 

-4.51 

-2.651061 

-4.19 

86 

EL4 

-0.432345 

-3.13 

-0.495506 

-2.45 

-0.360989 

-3.70 

-1.801613 

-3.49 

87 

EL5 

-0.411804 

-3.10 

-0.571936 

-2.60 

-0.414541 

-3.95 

-1.841014 

-3.51 

88 

FAl 

-0.394492 

-2.83 

-0.408074 

-2.19 

-0.357360 

-4.16 

-1.920122 

-3.75 

89 

GEl 

-0.451610 

-3.09 

-0.595829 

-2.58 

-0.516447 

-4.36 

-1.965759 

-3.57 

90 

GE2 

-0.508666 

-3.14 

-0.630420 

-2.62 

-0.855872 

-4.58 

-2.787292 

-4.22 

91 

MDl 

-0.230080 

-3.00 

-0.271103 

-1.79 

-0.083733 

-1.46 

-0.956490 

-2.43 

92 

MD2 

-0.219325 

-2.77 

-0.338773 

-2.03 

-0.065448 

-0.96 

-1.215833 

-2.77 

93 

Mil 

-0.305665 

-2.96 

-0.180606 

-1.29 

0.111301 

1.41 

-0.589841 

-1.48 

94 

MI2 

-0.182873 

-2.66 

-0.327408 

-1.99 

-0.160152 

-2.29 

-1.159399 

-2.68 

95 

MMl 

-0.522227 

-3.23 

-0.619509 

-2.67 

-0.592637 

-4.58 

-2.416817 

-3.89 

96 

MM2 

-0.546496 

-3.19 

-0.631461 

-2.72 

-0.537425 

-4.42 

-2.513423 

-4.08 

97 

MM3 

-0.791066 

-3.30 

-0.877795 

-3.01 

-0.739877 

-4.75 

-3.363766 

-4.45 

98 

PAl 

0.128144 

1.64 

-0.038525 

-0.30 

0.075406 

0.70 

-1.092372 

-2.44 

99 

PWl 

-0.340156 

-3.05 

-0.457278 

-2.36 

-0.370239 

-3.89 

-2.197317 

-3.94 

100 

TEl 

-0.357425 

-3.02 

-0.446044 

-2.35 

-0.328980 

-3.65 

-2.077871 

-3.87 

101 

TSl 

-0.333689 

-2.56 

-0.463253 

-2.36 

-0.323622 

-3.86 

-1.628818 

-3.21 

102 

VSl 

-0.123101 

-1.65 

-0.168930 

-1.14 

0.142695 

1.46 

-1.084590 

-2.39 

999 

0 

-1.213891 

-4.11 

-1 . 537020 

-4.56 

-1.198903 

-3.60 

-0.191798 

-0.31 
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Table  B.2.  Utility  Weights  and  Scale  Parameter  Estimates  by  Quarter 


Parameter 

First  Quarter 

Second  Quarter 

Third  Quarter 

Fourth  Quarter 

Estimate 

T-stat 

Estimate 

T-stat 

Estimate 

T-stat 

Estimate 

T-stat 

B  Rnk 

-0.001625 

-2.97 

-0.001554 

-3.24 

-0.002327 

-3.44 

-0.013694 

-3.57 

B  RrnkC 

0.000052 

3.32 

0.000036 

3.40 

0.000057 

3.78 

0.000318 

3.82 

B  IsTEAb 

0.007202 

1.59 

-0.000837 

-0.23 

-0.005652 

-0.97 

0.062023 

2.80 

B  SBd 

0.010549 

2.78 

0.003827 

1.99 

0.007371 

2.09 

0.102794 

3.28 

B  SBSd 

0.014734 

1.33 

0.013372 

1.17 

-0.041796 

-2.04 

-0.024582 

-0.24 

B  Abd 

0.009863 

2.21 

0.004969 

1.42 

0.011112 

1.54 

-0.028395 

-0.90 

B  HGd 

-0.003541 

-0.93 

-0.006322 

-1.63 

-0.010718 

-1.65 

-0.029531 

-1.57 

B_AA 

0.003662 

2.90 

0.003753 

3.08 

0.008209 

3.59 

0.048079 

5.22 

G  sexM2 

0.000000 

FIXED 

-0.173337 

-1.36 

0.000000 

FIXED 

-0.281300 

-0.88 

G  sexM3 

0.108766 

2.67 

0.074339 

2.60 

0.155695 

3.55 

0.401274 

2.90 

G  sexM5 

0.139172 

2.33 

0.193341 

2.81 

0.464723 

3.95 

0.568460 

2.41 

G  sexM6 

0.085558 

2.38 

0.152940 

3.41 

0.181201 

3.77 

0.476525 

2.96 

G  sexM7 

0.000000 

FIXED 

0.000000 

0.00 

0.000000 

FIXED 

-0.264854 

-1.98 

G  sexM999 

-0.264133 

-1.72 

-0.563628 

-3.97 

-0.555001 

-3.41 

-0.030834 

-0.14 

G  edNGl 

-0.091624 

-2.13 

-0.091409 

-2.39 

-0.115609 

-2.90 

-0.269677 

-1.53 

G  edNG3 

0.000000 

FIXED 

0.000000 

0.00 

0.000000 

FIXED 

0.285189 

1.84 

G  edNG'9 

0.075006 

2.22 

0.023052 

0.93 

0.078109 

2.31 

0.218497 

1.29 

G  edNGS 

-0.104742 

-1.70 

-0.097916 

-1.48 

0.000000 

FIXED 

-0 .488620 

-1.29 

G  edNG7 

0.074408 

2.16 

-0.044297 

-1.21 

0.000000 

FIXED 

0.187927 

1.11 

G  edNG9 

-0.111716 

-2 . 47 

0.000000 

0.00 

-0.118721 

-2.98 

-0.745136 

-3.46 

G_edNG999 

-0.348265 

-1.63 

0.022434 

0.11 

0.222336 

1.15 

0.783191 

2.54 

G  edSl 

0.000000 

FIXED 

0.000000 

0.00 

0.000000 

FIXED 

-0.308305 

-3.00 

G  edS3 

-0.096816 

-2.53 

0.000000 

0.00 

0.000000 

FIXED 

0.000000 

FIXED 

G  edS4 

-0.067197 

-1.86 

-0.095386 

-2.57 

-0.064835 

-1 . 69 

0.000000 

FIXED 

G  edS5 

0.000000 

FIXED 

0.000000 

0.00 

0.076672 

1.47 

0.128582 

0.77 

G_edS7 

-0.067159 

-1.90 

0.000000 

FIXED 

0 . 000000 

FIXED 

G_edS999 

0.561830 

3.25 

0.755945 

4.55 

0.465075 

2.64 

-0.297631 

-1.24 

G  edGCl 

0.000000 

FIXED 

0.000000 

0.00 

0.000000 

FIXED 

0.229713 

1.82 

G  edGC3 

0.000000 

FIXED 

0.000000 

0.00 

0.000000 

FIXED 

-0.119111 

-0.74 

G  edGC4 

-0.035313 

-0.94 

-0.076160 

-1.99 

0.000000 

FIXED 

-0.479138 

-2.15 

G  edGC5 

0.000000 

FIXED 

-0.056052 

-0.96 

0.000000 

FIXED 

-0.500291 

-1.32 

G  edGC6 

0.000000 

FIXED 

0.000000 

0.00 

-0.119665 

-2.56 

-0 . 318908 

-2.00 

G_edGC7 

0.060868 

1.71 

0.012656 

0.40 

G_edGC999 

0.674998 

3.50 

0.178732 

0.89 

0.079015 

0.34 

0.452194 

1.42 

G  AQA2 

0.000000 

FIXED 

0.000000 

0.00 

0.131126 

3.90 

0.000000 

FIXED 

G  AQA3 

-0.095033 

-2.13 

-0.071556 

-2.48 

-0.060232 

-1.53 

-0.453822 

-3.05 

G  AQA4 

0.000000 

FIXED 

-0.037582 

-1.70 

0.000000 

FIXED 

0.000000 

FIXED 

G  AQA5 

-0.120818 

-2.27 

-0.078144 

-1.73 

0.000000 

FIXED 

0.000000 

FIXED 

G  AQA6 

0.000000 

FIXED 

0.000000 

0.00 

-0.063617 

-2.11 

0.000000 

FIXED 

G  AQA9 

0 . 065356 

2.17 

0.000000 

0.00 

0.060464 

1.71 

0.000000 

FIXED 

G_AQA999 

0.014532 

0.07 

0.518593 

2.41 

0.181568 

0.77 

-0.092820 

-0.29 

G_RS999 

0.088162 

0.67 

-0.044285 

-0.34 

-0.096464 

-0.66 

0.043263 

0.22 

G_AFQT999 

-0.002293 

-0.43 

-0.004953 

-0.94 

0.003845 

0.61 

0.035553 

3.58 

LAMBDA 

8 . 859739 

1 . 82 

9.605288 

1.72 

7.358447 

1 . 03 

2 . 919374 

0.33 

DELTA 

0.831793 

-1.50 

0.857459 

-1.42 

0.753447 

-3.04 

0.535445 

-7.50 
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Table  C.l.  Fit  Diagnostics  for  First  Quarter  Model 


Subgroup 

Job 

Freq 

Estimate 

Actual 

Diff 

Ratio 

Family 

(E) 

(A) 

(A-E) 

(E/A) 

ALL 

CL 

3181 

0.1374 

0.1354 

-0.0020 

1 . 0145 

CO 

2656 

0.3691 

0.3698 

0.0007 

0.9980 

EL 

2517 

0.1210 

0.1211 

0.0000 

0.9998 

FA 

2584 

0.0974 

0.0989 

0.0014 

0.9857 

GM 

2051 

0.0414 

0.0427 

0.0013 

0.9699 

MM 

3125 

0.1311 

0.1310 

-0.0001 

1.0007 

OF 

2964 

0.0683 

0.0682 

-0.0001 

1 . 0008 

SC 

136 

0.2533 

0.2308 

-0.0225 

1 . 0974 

ST 

2942 

0.2218 

0.2220 

0.0002 

0.9993 

Non-Acc 

4080 

0.1687 

0.1688 

0.0002 

0.9990 

Multp-Opp 

CL 

3119 

0.1245 

0.1231 

-0.0014 

1.0117 

CO 

2537 

0.3412 

0.3409 

-0.0003 

1.0009 

EL 

2493 

0.1149 

0.1150 

0.0001 

0.9991 

FA 

2563 

0.0914 

0.0924 

0.0010 

0.9894 

GM 

2031 

0.0352 

0.0366 

0.0014 

0.9608 

MM 

3086 

0.1236 

0.1234 

-0.0002 

1 . 0017 

OF 

2946 

0.0643 

0.0639 

-0.0004 

1 . 0067 

SC 

131 

0.2333 

0.2177 

-0.0156 

1 . 0716 

ST 

2860 

0.2044 

0.2045 

0.0001 

0.9995 

Non-Acc 

3690 

0.1702 

0.1711 

0.0009 

0.9950 

Single-Opp 

CL 

62 

0.8230 

0.7928 

-0.0302 

1.0381 

CO 

119 

0.8730 

0.8928 

0.0198 

0.9779 

EL 

24 

0.8465 

0.8376 

-0.0089 

1 . 0107 

FA 

21 

0.8607 

0.9167 

0.0560 

0.9389 

GM 

20 

0.8033 

0.7859 

-0.0173 

1.0221 

MM 

39 

0.8327 

0.8431 

0.0104 

0.9877 

OF 

18 

0.8198 

0.8900 

0.0702 

0.9211 

SC 

5 

0.8226 

0.6038 

-0.2188 

1.3624 

ST 

82 

0.8300 

0.8323 

0.0023 

0.9973 

Non-Acc 

390 

0.1542 

0.1480 

-0.0062 

1.0421 

Male 

CL 

2545 

0.1051 

0.0974 

-0.0077 

1.0794 

CO 

2463 

0.3947 

0.3953 

0.0006 

0.9985 

EL 

2048 

0.1205 

0.1195 

-0.0009 

1.0076 

FA 

2497 

0.1001 

0.1018 

0.0017 

0.9836 

GM 

1526 

0.0406 

0.0426 

0.0020 

0.9537 

MM 

2541 

0.1369 

0.1367 

-0.0002 

1.0014 

OF 

2409 

0.0616 

0.0625 

0.0009 

0.9853 

SC 

110 

0.2301 

0.2283 

-0.0018 

1 . 0077 

ST 

2318 

0.1715 

0.1759 

0.0044 

0.9751 

Non-Acc 

3259 

0.1570 

0.1573 

0.0003 

0.9980 

Female 

CL 

636 

0.2733 

0.2956 

0.0223 

0.9246 

CO 

193 

0.0138 

0.0167 

0.0029 

0.8271 

EL 

469 

0.1237 

0.1281 

0.0044 

0.9654 

FA 

87 

0.0116 

0.0050 

-0.0066 

2.3098 

GM 

525 

0.0437 

0.0428 

-0.0009 

1.0200 

MM 

584 

0.1043 

0.1047 

0.0003 

0.9968 

OF 

555 

0.0991 

0.0946 

-0.0045 

1.0481 

SC 

26 

0.3449 

0.2407 

-0.1042 

1 .4328 

ST 

624 

0.4215 

0.4050 

-0.0165 

1 . 0408 

Non-Acc 

821 

0.2175 

0.2170 

-0.0004 

1.0021 
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Table  C.l.  Fit  Diagnostics  for  First  Quarter  Model  (continued 


Subgroup 

Job 

Family 

Freq 

Estimate 

(E) 

Actual 

(A) 

Diff 

(A-E) 

Ratio 

(E/A) 

HSGC 

CL 

418 

0.1115 

0.1066 

0.0050 

1.0465 

CO 

364 

0.2955 

0.3071 

0.0116 

0.9621 

EL 

365 

0.1247 

0.1016 

0.0231 

1.2275 

FA 

332 

0.0572 

0.0578 

0.0007 

0.9887 

GM 

290 

0.0233 

0.0286 

0.0053 

0.8145 

MM 

417 

0.0914 

0.0815 

0.0099 

1.1212 

OF 

399 

0.0637 

0.0654 

0.0017 

0.9736 

SC 

15 

0.2355 

0.1714 

0.0641 

1.3741 

ST 

433 

0.2609 

0.2783 

0.0174 

0.9374 

Non-Acc 

533 

0.2272 

0.2298 

0.0025 

0.9890 

HSG 

CL 

1775 

0.1466 

0.1514 

0.0048 

0.9680 

CO 

1436 

0.3887 

0.3854 

0.0033 

1.0086 

EL 

1301 

0.1229 

0.1277 

0.0049 

0.9619 

FA 

1384 

0.0959 

0.0980 

0.0021 

0.9787 

GM 

1129 

0.0435 

0.0438 

0.0003 

0.9935 

MM 

1725 

0.1306 

0.1312 

0.0005 

0.9959 

OF 

1624 

0.0610 

0.0607 

0.0003 

1 . 0052 

SC 

37 

0.3049 

0.3145 

0.0097 

0.9693 

ST 

1537 

0.2232 

0.2152 

0.0080 

1.0372 

Non-Acc 

2187 

0.1510 

0.1503 

0.0007 

1.0049 

Senior 

CL 

592 

0.1539 

0.1277 

0.0262 

1.2053 

CO 

4  68 

0.3107 

0.3160 

0.0053 

0.9832 

EL 

4  97 

0.0949 

0.0929 

0.0020 

1.0213 

FA 

490 

0.0619 

0.0621 

0.0002 

0.9970 

GM 

337 

0.0502 

0.0512 

0.0010 

0.9803 

MM 

543 

0.0951 

0.1020 

0.0069 

0.9324 

OF 

546 

0.0541 

0.0549 

0.0007 

0.9867 

SC 

55 

0.3083 

0.3060 

0.0022 

1.0073 

ST 

577 

0.2378 

0.2529 

0.0151 

0.9404 

Non-Acc 

733 

0.2207 

0.2215 

0.0008 

0.9962 

NG 

CL 

396 

0.0966 

0.1046 

0.0080 

0.9237 

CO 

388 

0.4382 

0.4380 

0.0002 

1 . 0004 

EL 

354 

0.1487 

0.1583 

0.0096 

0.9393 

FA 

378 

0.1884 

0.1897 

0.0012 

0.9935 

GM 

295 

0.0408 

0.0423 

0.0015 

0.9648 

MM 

440 

0.2193 

0.2173 

0.0020 

1 . 0094 

OF 

395 

0.1254 

0.1234 

0.0020 

1 . 0162 

SC 

29 

0.0990 

0.0237 

0.0753 

4 . 1852 

ST 

395 

0.1462 

0.1364 

0.0098 

1.0717 

Non-Acc 

627 

0.1174 

0.1180 

0.0006 

0.9953 
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Table  C.l.  Fit  Diagnostics  for  First  Quarter  Model  (continued) 


Subgroup 

Job 

Freq 

Estimate 

Actual 

Diff 

Ratio 

Family 

(E) 

(A) 

(A-E) 

(E/A) 

Catl-3A 

CL 

2434 

0.1070 

0.1048 

-0.0021 

1.0204 

CO 

2148 

0.3512 

0.3605 

0.0093 

0.9741 

EL 

2256 

0.1207 

0.1213 

0.0006 

0.9952 

FA 

2089 

0.0953 

0.0945 

-0.0008 

1.0087 

GM 

1709 

0.0283 

0.0284 

0.0002 

0.9942 

MM 

2491 

0.1194 

0.1159 

-0.0036 

1.0308 

OF 

2359 

0.0592 

0.0553 

-0.0039 

1.0702 

SC 

83 

0.2376 

0.3312 

0.0936 

0.7174 

ST 

2466 

0.2282 

0.2259 

-0.0022 

1.0099 

Non-Acc 

3079 

0.1618 

0.1621 

0.0004 

0.9977 

Cat3B 

CL 

747 

0.2357 

0.2343 

-0.0014 

1.0060 

CO 

508 

0.4426 

0.4079 

-0.0347 

1.0850 

EL 

261 

0.1234 

0.1187 

-0.0047 

1.0397 

FA 

495 

0.1066 

0 .1175 

0.0109 

0.9071 

GM 

342 

0.1080 

0.1150 

0.0070 

0 . 9392 

MM 

634 

0.1769 

0.1904 

0.0135 

0 . 9289 

OF 

605 

0.1030 

0 . 1176 

0.0146 

0 . 8762 

SC 

53 

0.2751 

0.0918 

-0.1833 

2.9975 

ST 

476 

0.1901 

0.2022 

0.0122 

0 . 9399 

Non-Acc 

1001 

0.1896 

0.1891 

-0.0005 

1.0025 
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Table  C.2.  Fit  Diagnostics  for  Second  Quarter  Model 


Subgroup 

Job 

Freq 

Estimate 

Actual 

Diff 

Ratio 

Family 

nni 

(A) 

(A-E) 

(E/A) 

ALL 

CL 

3419 

0.1673 

0.1657 

-0.0017 

1.0100 

CO 

3069 

0.3650 

0.3633 

-0.0017 

1.0048 

EL 

2826 

0.1279 

0.1289 

0.0010 

0.9923 

FA 

2631 

0.1078 

0.1067 

-0.0010 

1.0097 

GM 

1723 

0.0324 

0.0351 

0.0027 

0.9225 

MM 

3226 

0.1303 

0.1324 

0.0021 

0.9840 

OF 

3040 

0.0550 

0.0552 

0.0002 

0.9960 

SC 

229 

0.1683 

0.1664 

-0.0019 

1.0116 

ST 

3251 

0.2156 

0.2154 

-0.0002 

1.0012 

Non-Acc 

4389 

0.1471 

0.1472 

0.0001 

0.9995 

Mult-Opp 

CL 

3343 

0.1528 

0.1502 

-0.0025 

1.0169 

CO 

2901 

0.3328 

0.3323 

-0.0006 

1.0018 

EL 

2800 

0.1225 

0.1225 

0.0000 

1.0000 

FA 

2602 

0.0999 

0.0994 

-0.0006 

1.0055 

GM 

1704 

0.0279 

0.0306 

0.0027 

0.9133 

MM 

3187 

0.1228 

0.1240 

0.0012 

0.9906 

OF 

3015 

0.0497 

0.0494 

-0.0003 

1.0067 

SC 

224 

0.1525 

0.1551 

0.0026 

0.9832 

ST 

3157 

0.1959 

0.1956 

-0.0003 

1.0017 

Non-Acc 

3908 

0.1483 

0.1496 

0.0013 

0.9913 

Single-Opp 

CL 

76 

0.8485 

0.8880 

0.0394 

0.9556 

CO 

168 

0.8798 

0.8596 

-0.0202 

1.0235 

EL 

26 

0.8653 

1.0000 

0.1347 

0.8653 

FA 

29 

0.8686 

0.8199 

-0.0487 

1.0594 

GM 

19 

0.8156 

0.8311 

0.0155 

0.9813 

MM 

39 

0.8553 

0.9505 

0.0953 

0.8998 

OF 

25 

0.8384 

0.9207 

0.0823 

0.9106 

SC 

5 

0.8700 

0.6672 

-0.2028 

1.3040 

ST 

94 

0.8496 

0.8522 

0.0027 

0.9969 

Non-Acc 

481 

0.1375 

0.1273 

-0.0102 

1.0803 

Male 

CL 

2701 

0.1263 

0.1188 

-0.0074 

1.0627 

CO 

2800 

0.3981 

0.3965 

-0.0016 

1.0041 

EL 

2273 

0.1268 

0.1278 

0.0010 

0.9919 

FA 

2562 

0.1097 

0.1083 

-0.0014 

1.0131 

GM 

1213 

0.0374 

0.0398 

0.0024 

0.9389 

MM 

2597 

0.1433 

0.1450 

0.0017 

0.9884 

OF 

2416 

0.0450 

0.0451 

0.0000 

0.9990 

SC 

131 

0.1568 

0.1966 

0.0398 

0.7977 

ST 

2563 

0.1686 

0.1744 

0.0058 

0.9670 

Non-Acc 

3475 

0.1298 

0.1295 

-0.0002 

1.0019 

Female 

CL 

718 

0.3193 

0.3391 

0.0197 

0.9418 

CO 

269 

0.0149 

0.0120 

-0.0028 

1.2346 

EL 

553 

0.1327 

0.1335 

0.0008 

0.9937 

FA 

69 

0.0282 

0.0426 

0.0143 

0.6634 

GM 

510 

0.0202 

0.0236 

0.0034 

0.8548 

MM 

629 

0.0760 

0.0800 

0.0040 

0.9504 

OF 

624 

0.0927 

0.0936 

0.0009 

0.9905 

SC 

98 

0.1829 

0.1280 

-0.0549 

1.4293 

ST 

688 

0.3908 

0.3681 

-0.0226 

1.0615 

Non-Acc 

914 

0.2134 

0.2147 

0.0013 

0.9940 
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Table  C.2.  Fit  Diagnostics  for  Second  Quarter  Model  (continued) 


Subgroup 

Job 

Freq 

Estimate 

Actual 

Diff 

Ratio 

Family 

(E) 

(A) 

(A-E) 

(E/A) 

HSGC 

CL 

429 

0.1640 

0.1729 

0.0089 

0.9486 

CO 

376 

0.3280 

0.3116 

-0.0164 

1.0528 

EL 

384 

0.1406 

0.1525 

0.0119 

0.9220 

FA 

335 

0.0681 

0.0620 

-0.0061 

1.0987 

GM' 

244 

0.0191 

0.0208 

0.0017 

0.9181 

MM 

421 

0.1059 

0.0986 

-0.0074 

1.0747 

OF 

406 

0.0577 

0.0577 

0.0000 

0.9997 

SC 

24 

0.3087 

0.2744 

-0.0343 

1.1249 

ST 

469 

0.2827 

0.2915 

0.0089 

0.9696 

Non-Acc 

567 

0.1468 

0.1455 

-0.0013 

1.0089 

HSG 

CL 

1964 

0.1713 

0.1767 

0.0054 

0.9692 

CO 

1751 

0.3613 

0.3586 

-0.0027 

1.0076 

EL 

1521 

0.1167 

0.1162 

-0.0006 

1 . 0048 

FA 

1514 

0.1154 

0.1159 

0.0005 

0.9957 

GM 

993 

0.0331 

0.0380 

0.0050 

0.8697 

MM 

1813 

0.1311 

0.1335 

0.0024 

0.9818 

OF 

1681 

0.0593 

0.0597 

0.0005 

0.9921 

SC 

91 

0.1676 

0.1448 

-0.0227 

1.1571 

ST 

1799 

0.2025 

0.1950 

-0.0076 

1 . 0388 

Non-Acc 

2378 

0.1229 

0.1228 

-0.0001 

1.0006 

Senior 

CL 

656 

0.1856 

0.1540 

-0.0316 

1.2055 

CO 

568 

0.3300 

0.3445 

0.0145 

0.9580 

EL 

572 

0.1310 

0.1097 

-0.0213 

1.1939 

FA 

473 

0.0515 

0.0529 

0.0013 

0.9749 

GM 

284 

0.0489 

0.0457 

-0.0032 

1.0705 

MM 

623 

0.1075 

0.1220 

0.0145 

0.8812 

OF 

613 

0.0397 

0.0376 

-0.0021 

1.0562 

SC 

92 

0.1367 

0.1756 

0.0389 

0.7785 

ST 

635 

0.2070 

0.2274 

0.0205 

0.9100 

Non-Acc 

852 

0.2221 

0.2232 

0.0010 

0.9953 

NG 

CL 

370 

0.1154 

0.1202 

0.0048 

0.9600 

CO 

374 

0.4708 

0.4623 

-0.0085 

1 . 0184 

EL 

349 

0.1595 

0.1961 

0.0367 

0.8131 

FA 

309 

0.2011 

0.1942 

-0.0069 

1.0353 

GM 

202 

0.0204 

0.0221 

0.0017 

0.9229 

MM 

369 

0.1957 

0.1841 

-0.0116 

1 . 0632 

OF 

340 

0.0597 

0.0634 

0.0037 

0.9413 

SC 

22 

0.1686 

0.1057 

-0.0629 

1.5949 

ST 

348 

0.2136 

0.1994 

-0.0142 

1.0713 

Non-Acc 

592 

0.1314 

0.1318 

0.0005 

0.9964 
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Table  C.2.  Fit  Diagnostics  for  Second  Quarter  Model  (continued 


Subgroup 

Job 

Family 

Freq 

Estimate 

(E) 

Actual 

(A) 

Diff 

(A-E) 

Ratio 

(E/A) 

Catl-3A 

CL 

2408 

0.1304 

0.1186 

-0.0118 

1.0998 

CO 

2270 

0.3320 

0.3384 

0.0064 

0.9812 

EL 

2351 

0.1332 

0.1340 

0.0009 

0.9935 

FA 

2048 

0.1002 

0.0986 

■0.0016 

1.0164 

GM 

1342 

0.0216 

0.0241 

0.0025 

0.8963 

MM 

2473 

0.1195 

0.1165 

■0.0030 

1.0260 

OF 

2324 

0.0445 

0.0461 

0.0016 

0.9652 

SC 

143 

0.1861 

0.2023 

0.0162 

0.9201 

ST 

2554 

0.2202 

0.2248 

0.0046 

0.9794 

Non-Acc 

3110 

0.1549 

0.1553 

0.0005 

0.9970 

Cat3B 

CL 

974 

0.2517 

0.2740 

0.0223 

0.9186 

CO 

769 

0.4545 

0.4209  •  - 

■0.0336 

1.0799 

EL 

4  68 

0.1026 

0.1051 

0.0025 

0.9764 

FA 

577 

0.1338 

0.1355 

0.0017 

0.9873 

GM 

362 

0.0681 

0.0730 

0.0049 

0.9330 

MM 

729 

0.1619 

0.1816 

0.0198 

0.8912 

OF 

684 

0.0844 

0.0833 

■0.0011 

1.0136 

SC 

84 

0.1357 

0.1043 

■0.0314 

1 . 3016 

ST 

692 

0.2001 

0.1830 

■0.0171 

1.0934 

Non-Acc 

1230 

0.1291 

0.1303 

0.0011 

0.9912 

Cat4 

CL 

37 

0.2943 

0.3072 

0.0129 

0.9581 

CO 

30 

0.5390 

0.7608 

0.2218 

0.7085 

EL 

7 

0.0562 

0.0000 

■0.0562 

FA 

6 

0.1607 

0.0810 

0.0796 

1 . 9832 

GM 

19 

0.1212 

0.0960 

■0.0252 

1.2622 

MM 

24 

0.2800 

0.2737 

■0.0063 

1.0231 

OF 

32 

0.1750 

0.1014 

0.0736 

1.7263 

SC 

2 

0.1615 

0.0000 

0.1615 

, 

ST 

5 

0.0953 

0.0000 

■0.0953 

, 

Non-Acc 

49 

0.1195 

0.0665 

0.0530 

1.7979 

Table  C.3.  Fit  Diagnostics  for  Third  Quarter  Model 


Subgroup 

Job 

Freq 

Estimate 

Actual 

Diff 

Ratio 

Family 

(E) 

(A) 

(A-E) 

(E/A) 

ALL 

CL 

3400 

0.1967 

0.1945 

-0.0023 

1.0116 

CO 

2511 

0.3321 

0.3322 

0.0001 

0.9998 

EL 

2493 

0.1423 

0.1430 

0.0007 

0.9952 

FA 

2561 

0.1260 

0.1249 

-0.0012 

1.0093 

GM 

1759 

0.0349 

0.0331 

-0.0018 

1 . 0539 

MM 

3213 

0.1607 

0.1632 

0.0025 

0.9846 

OF 

3200 

0.0936 

0.0941 

0.0005 

0.9947 

SC 

170 

0.1512 

0.1488 

-0.0024 

1 . 0163 

ST 

2898 

0.2256 

0.2268 

0.0013 

0. 9944 

Non-Acc 

4394 

0.1541 

0.1539 

-0.0002 

1 . 0015 

Mult-Opp 

CL 

3324 

0.1811 

0.1786 

-0.0026 

1 . 0143 

CO 

2380 

0.3057 

0.3050 

-0.0007 

1 . 0023 

EL 

2471 

0.1367 

0.1370 

0.0003 

0.9977 

FA 

2541 

0.1191 

0.1199 

0.0009 

0.9929 

GM 

1743 

0.0310 

0.0295 

-0.0015 

1.0510 

MM 

3178 

0.1535 

0.1555 

0.0020 

0.9874 

OF 

3168 

0.0856 

0.0862 

0.0006 

0.9928 

SC 

166 

0.1404 

0.1359 

-0.0044 

1 . 0327 

ST 

2808 

0.2055 

0.2070 

0.0015 

0.9927 

Non-Acc 

3968 

0.1565 

0.1560 

-0.0005 

1.0030 

Single-Opp 

CL 

76 

0.8603 

0.8711 

0.0108 

0.9876 

CO 

131 

0.8918 

0.9084 

0.0166 

0.9817 

EL 

22 

0.8630 

0.9113 

0.0483 

0.9470 

FA 

20 

0.8627 

0.6484 

-0.2143 

1 . 3305 

GM 

16 

0.8543 

0.7936 

-0.0607 

1 . 0765 

MM 

35 

0.8418 

0.8984 

0.0566 

0.9370 

OF 

32 

0.8593 

0.8473 

-0.0120 

1.0141 

SC 

4 

0.8693 

1.0000 

0.1307 

0.8693 

ST 

90 

0.8711 

0.8641 

-0.0070 

1 . 0081 

Non-Acc 

426 

0.1300 

0.1321 

0.0022 

0.9835 

Male 

CL 

2772 

0.1613 

0.1590 

-0.0023 

1.0146 

CO 

2441 

0.3407 

0.3411 

0.0004 

0.9988 

EL 

1939 

0.1381 

0.1368 

-0.0013 

1.0098 

FA 

2527 

0.1273 

0.1263 

-0.0010 

1.0076 

GM 

1278 

0.0408 

0.0378 

-0.0031 

1.0809 

MM 

2606 

0.1684 

0.1705 

0.0021 

0.9874 

OF 

2660 

0.0864 

0.0907 

0.0044 

0.9520 

SC 

119 

0.1042 

0.1350 

0.0308 

0.7722 

ST 

2318 

0.1923 

0.1902 

-0.0021 

1.0112 

Non-Acc 

3521 

0.1377 

0.1373 

-0.0004 

1 . 0026 

Female 

CL 

628 

0.3498 

0.3478 

-0.0020 

1.0057 

CO 

70 

0.0299 

0.0175 

-0.0124 

1.7089 

EL 

554 

0.1567 

0.1644 

0.0077 

0.9533 

FA 

34 

0.0182 

0.0000 

-0.0182 

GM 

481 

0.0190 

0.0206 

0.0016 

0.9209 

MM 

607 

0.1272 

0.1314 

0.0042 

0.9683 

OF 

540 

0.1284 

0.1103 

-0.0181 

1.1644 

SC 

51 

0.2618 

0.1812 

-0.0806 

1.4447 

ST 

580 

0.3573 

0.3720 

0.0147 

0.9604 

Non-Acc 

873 

0.2194 

0.2196 

0.0003 

0.9987 
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Table  C.3.  Fit  Diagnostics  for  Third  Quarter  Model  (continued] 


Subgroup 

Job 

Freq 

Estimate 

Actual 

Diff 

Ratio 

Family 

(A) 

(A-E) 

(E/A) 

HSGC 

CL 

434 

0.2033 

0.1984 

-0.0050 

1.0250 

CO 

287 

0.3156 

0.2769 

-0.0387 

1.1396 

EL 

317 

0.1580 

0.1569 

-0.0011 

1.0069 

FA 

297 

0.0942 

0.1097 

0.0154 

0.8592 

GM 

253 

0.0245 

0.0220 

-0.0025 

1.1122 

MM 

403 

0.0821 

0.0821 

-0.0001 

1.0008 

OF 

391 

0.0722 

0.0622 

-0.0100 

1.1606 

SC 

13 

0.1631 

0.3186 

0.1554 

0.5121 

ST 

409 

0.3152 

0.3431 

0.0279 

0.9186 

Non-Acc 

538 

0.1506 

0.1518 

0.0012 

0.9922 

HSG 

CL 

1665 

0.2198 

0.2198 

0.0000 

1.0000 

CO 

1162 

0.3069 

0.3285 

0.0217 

0.9341 

EL 

1149 

0.1410 

0.1395 

-0.0015 

1.0107 

FA 

1241 

0.1204 

0.1157 

-0.0046 

1.0401 

GM 

837 

0.0313 

0.0337 

0.0023 

0.9312 

MM 

1532 

0.1677 

0.1702 

0.0025 

0.9853 

OF 

1551 

0.0965 

0.0991 

0.0026 

0.9735 

SC 

102 

0.1602 

0.1345 

-0.0258 

1.1917 

ST 

1326 

0.2332 

0.2143 

-0.0190 

1.0884 

Non-Acc 

2064 

0.1366 

0.1370 

0.0004 

0.9974 

Senior 

CL 

747 

0.1822 

0.1687 

-0.0135 

1.0798 

CO 

651 

0.3785 

0.3581 

-0.0203 

1.0568 

EL 

618 

0.1319 

0.1543 

0.0224 

0.8546 

FA 

553 

0.0738 

0.0731 

-0.0007 

1.0092 

GM 

383 

0.0427 

0.0363 

-0.0064 

1.1755 

MM 

707 

0.1210 

0.1148 

-0.0063 

1.0546 

OF 

691 

0.0683 

0.0629 

-0.0054 

1.0856 

SC 

22 

0.1431 

0.2525 

0.1095 

0.5665 

ST 

680 

0.2117 

0.2384 

0.0267 

0.8880 

Non-Acc 

981 

0.1865 

0.1866 

0.0001 

0.9993 

NG 

CL 

554 

0.1425 

0.1506 

0.0080 

0.9467 

CO 

411 

0.3399 

0.3396 

-0.0002 

1.0007 

EL 

409 

0.1497 

0.1260 

-0.0237 

1.1878 

FA 

470 

0.2192 

0.2164 

-0.0028 

1.0130 

GM 

286 

0.0441 

0.0372 

-0.0069 

1.1853 

MM 

571 

0.2440 

0.2591 

0.0150 

0.9419 

OF 

567 

0.1295 

0.1381 

0.0086 

0.9376 

SC 

33 

0.1256 

0.0642 

-0.0615 

1 . 9575 

ST 

483 

0.1520 

0.1510 

-0.0010 

1.0068 

Non-Acc 

811 

0.1615 

0.1585 

-0.0030 

1.0189 
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Table  C.3.  Fit  Diagnostics  for  Third  Quarter  Model  (continued) 


Subgroup 

Job 

Freq 

Estimate 

Actual 

Diff 

Ratio 

Family 

(E) 

(A) 

(A-E) 

(E/A) 

Catl-3A 

CL 

2354 

0.1538 

0.1543 

0.0005 

0.9964 

CO 

1720 

0.3156 

0.3130 

-0.0027 

1.0085 

EL 

2042 

0.1453 

0.1454 

0.0001 

0.9992 

FA 

1952 

0.1088 

0.1084 

-0.0004 

1.0040 

GM 

1394 

0.0282 

0.0274 

-0.0007 

1.0273 

MM 

2462 

0.1334 

0.1345 

0.0011 

0.9916 

OF 

2364 

0.0801 

0.0769 

-0.0032 

1.0417 

SC 

88 

0.1896 

0.2234 

0.0338 

0.8488 

ST 

2336 

0.2406 

0.2443 

0.0037 

0.9848 

Non-Acc 

3017 

0.1544 

0.1539 

-0.0005 

1 . 0035 

Cat3B 

CL 

977 

0.2902 

0.2795 

-0.0107 

1 . 0382 

CO 

722 

0.3599 

0.3587 

-0.0012 

1.0032 

EL 

427 

0.1308 

0.1363 

0.0055 

0.9598 

FA 

579 

0.1814 

0.1782 

-0.0032 

1.0181 

GM 

326 

0.0580 

0.0559 

-0.0021 

1.0381 

MM 

692 

0.2336 

0.2550 

0.0214 

0.9162 

OF 

771 

0.1289 

0.1387 

0.0098 

0.9293 

SC 

80 

0.1125 

0.0722 

-0.0404 

1.5592 

ST 

544 

0.1654 

0.1541 

-0.0112 

1 . 0728 

Non-Acc 

1269 

0.1552 

0.1538 

-0.0014 

1 . 0091 

Cat4 

CL 

69 

0.2536 

0.2808 

0.0272 

0.9031 

CO 

69 

0.4591 

0.5438 

0.0847 

0.8443 

EL 

24 

0.0931 

0.0514 

-0.0417 

1 . 8099 

FA 

30 

0.1301 

0.1235 

-0.0066 

1.0536 

GM 

39 

0.0914 

0.0537 

-0.0377 

1.7027 

MM 

59 

0.4046 

0.2404 

-0.1642 

1 . 6831 

OF 

65 

0.1472 

0.1664 

0.0192 

0.884  6 

SC 

2 

0.0455 

0.0000 

-0.0455 

, 

ST 

18 

0.1776 

0.2485 

0.0709 

0.7145 

Non-Acc 

108 

0.1320 

0.1541 

0.0221 

0.8564 
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Table  C.4.  Fit  Diagnostics  for  Fourth  Quarter  Model 


Subgroup 

Job 

Freq 

Estimate 

Actual 

Diff 

Ratio 

Family 

(E) 

(A) 

(A-E) 

(E/A) 

ALL 

CL 

3183 

0.1882 

0.1917 

0.0035 

0.9817 

CO 

2962 

0.3377 

0.3375 

-0.0001 

1.0004 

EL 

2670 

0.1221 

0.1220 

-0.0001 

1.0005 

FA 

2612 

0.0952 

0.0935 

-0.0018 

1.0187 

GM 

2474 

0.0313 

0.0313 

0.0000 

1.0007 

MM 

3258 

0.1472 

0.1465 

-0.0007 

1.0047 

OF 

3110 

0.0773 

0.0776 

0.0003 

0.9967 

SC 

475 

0.1835 

0.1808 

-0.0026 

1.0146 

ST 

3253 

0.2526 

0.2509 

-0.0017 

1.0069 

Non-Acc 

4420 

0.1249 

0.1254 

0.0006 

0.9956 

Mult-Opp 

CL 

3128 

0.1761 

0.1784 

0.0023 

0.9869 

CO 

2835 

0.3118 

0.3118 

0.0000 

0.9998 

EL 

2645 

0.1152 

0.1153 

0.0001 

0.9993 

FA 

2597 

0.0912 

0.0893 

-0.0019 

1.0214 

GM 

2462 

0.0290 

0.0288 

-0.0002 

1.0063 

MM 

3225 

0.1408 

0.1397 

-0.0011 

1.0081 

OF 

3086 

0.0716 

0.0712 

-0.0004 

1.0055 

SC 

470 

0.1747 

0.1715 

-0.0032 

1.0189 

ST 

3168 

0.2325 

0.2324 

-0.0001 

1.0005 

Non-Acc 

4039 

0.1262 

0.1273 

0.0011 

0.9913 

Single-Opp 

CL 

55 

0.8799 

0.9496 

0.0697 

0.9266 

CO 

127 

0.9365 

0.9318 

-0.0047 

1.0050 

EL 

25 

0.8481 

0.8326 

-0.0155 

1.0186 

FA 

15 

0.7886 

0.8141 

0.0256 

0.9686 

GM 

12 

0.7692 

0.8198 

0.0506 

0.9383 

MM 

33 

0.8168 

0.8625 

0.0457 

0.9470 

OF 

24 

0.8043 

0.8863 

0.0820 

0.9075 

SC 

5 

0.9494 

1.0000 

0.0506 

0.9494 

ST 

85 

0.9117 

0.8571 

-0.0546 

1.0637 

Non-Acc 

381 

0.1107 

0.1054 

-0.0052 

1.0497 

Male 

CL 

2515 

0.1360 

0.1330 

-0.0030 

1.0228 

CO 

2731 

0.3649 

0.3647 

-0.0002 

1.0005 

EL 

2118 

0.1245 

0.1245 

0.0000 

1.0002 

FA 

2509 

0.0987 

0.0969 

-0.0018 

1.0186 

GM 

1894 

0.0300 

0.0302 

0.0003 

0.9902 

MM 

2640 

0.1594 

0.1589 

-0.0005 

1.0031 

OF 

2486 

0.0608 

0.0616 

0.0008 

0.9870 

SC 

269 

0.1505 

0.1900 

0.0395 

0.7922 

ST 

2523 

0.2040 

0.2038 

-0.0002 

1.0010 

Non-Acc 

3425 

0.1154 

0.1157 

0.0003 

0.9970 

Female 

CL 

668 

0.3799 

0.4074 

0.0275 

0.9326 

CO 

231 

0.0252 

0.0255 

0.0003 

0.9865 

EL 

552 

0.1128 

0.1126 

-0.0002 

1.0019 

FA 

103 

0.0095 

0.0089 

-0.0006 

1.0638 

GM 

580 

0.0357 

0.0346 

-0.0010 

1.0301 

MM 

618 

0.0957 

0.0941 

-0.0016 

1.0166 

OF 

624 

0.1413 

0.1394 

-0.0019 

1.0134 

SC 

206 

0.2265 

0.1689 

-0.0576 

1.3410 

ST 

730 

0.4147 

0.4078 

-0.0069 

1 . 0169 

Non-Acc 

995 

0.1567 

0.1580 

0.0013 

0.9920 
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Table  C.4.  Fit  Diagnostics  for  Fourth  Quarter  Model  (continued) 


Subgroup 

Job 

Freq 

Estimate 

Actual 

Diff 

Ratio 

Family 

(E) 

(A) 

(A-E) 

(E/A) 

HSGC 

CL 

334 

0.2078 

0.2008 

-0.0070 

1.0349 

CO 

304 

0.2969 

0.2594 

-0.0375 

1 . 1444 

EL 

277 

0.0932 

0.0949 

0.0017 

0.9822 

FA 

278 

0.0508 

0.0485 

-0.0022 

1.0456 

GM 

262 

0.0126 

0.0127 

0.0001 

0.9915 

MM 

340 

0.0829 

0.0839 

0.0010 

0.9878 

OF 

319 

0.0607 

0.0565 

-0.0042 

1 . 0739 

SC 

41 

0.2420 

0.3139 

0.0720 

0.7708 

ST 

378 

0.3335 

0.3656 

0.0321 

0.9122 

Non-Acc 

462 

0.1559 

0.1545 

-0.0013 

1.0085 

HSG 

CL 

1660 

0.2046 

0.2145 

0.0099 

0.9538 

CO 

1507 

0.3254 

0.3093 

-0.0161 

1 . 0521 

EL 

1280 

0.1109 

0.1242 

0.0133 

0.8929 

FA 

1395 

0.1001 

0.1019 

0.0018 

0.9826 

GM 

1364 

0.0360 

0.0363 

0.0002 

0.9936 

MM 

1669 

0.1470 

0.1473 

0.0003 

0.9981 

OF 

1574 

0.0802 

0.0842 

0.0040 

0.9524 

SC 

226 

0.1998 

0.1483 

-0.0515 

1.3470 

ST 

1682 

0.2612 

0.2548 

-0.0064 

1.0250 

Non-Acc 

2286 

0.1210 

0.1225 

0.0015 

0.9877 

Senior 

CL 

946 

0.1696 

0.1667 

-0.0029 

1.0175 

CO 

893 

0.3728 

0.4171 

o 

o 

0.8937 

EL 

864 

0.1307 

0.1099 

-0.0208 

1.1888 

FA 

713 

0.0789 

0.0679 

-0.0110 

1.1616 

GM 

649 

0.0319 

0.0316 

-0.0003 

1.0096 

MM 

970 

0.1581 

0.1531 

-0.0050 

1.0325 

OF 

965 

0.0700 

0.0644 

-0.0056 

1.0864 

SC 

175 

0.1530 

0.1940 

0.0411 

0.7883 

ST 

924 

0.2416 

0.2324 

-0.0093 

1.0398 

Non-Acc 

1277 

0.1012 

0.1019 

0.0007 

0.9933 

NG 

CL 

243 

0.1222 

0.1211 

-0.0010 

1.0086 

CO 

258 

0.3409 

0.3297 

-0.0111 

1.0337 

EL 

249 

0.1819 

0.1810 

-0.0009 

1.0048 

FA 

226 

0.1692 

0.1743 

0.0051 

0.9706 

GM 

199 

0.0228 

0.0217 

-0.0010 

1.0468 

MM 

279 

0.1893 

0.1951 

0.0058 

0.9703 

OF 

252 

0.1078 

0.1120 

0.0042 

0.9625 

SC 

33 

0.1569 

0.1491 

-0.0079 

1 . 0528 

ST 

269 

0.1245 

0.1288 

0.0043 

0.9665 

Non-Acc 

395 

0.1841 

0.1810 

-0.0031 

1.0169  1 
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Table  C.4.  Fit  Diagnostics  for  Fourth  Quarter  Model  (continued) 


Subgroup 

Job 

Freq 

Estimate 

Actual 

Diff 

Ratio 

Family 

(E) 

(A) 

(A-E) 

(E/A) 

Catl-3A 

CL 

2319 

0.1419 

0.1465 

0.0046 

0.9687 

CO 

2151 

0.3059 

0.3219 

0.0160 

0.9503 

EL 

2205 

0.1206 

0.1201 

-0.0004 

1.0036 

FA 

2055 

0.0924 

0.0867 

-0.0057 

1.0659 

GM 

1921 

0.0255 

0.0235 

-0.0021 

1.0875 

MM 

2534 

0.1328 

0.1243 

-0.0085 

1.0684 

OF 

2314 

0.0623 

0.0577 

-0.0046 

1.0795 

SC 

294 

0.2267 

0.2416 

0.0149 

0.9385 

ST 

2633 

0.2701 

0.2686 

-0.0015 

1.0057 

Non-Acc 

3133 

0.1236 

0.1247 

0.0010 

0.9916 

Cat3B 

CL 

859 

0.3121 

0.3123 

0.0001 

0.9995 

CO 

809 

0.4226 

0.3779 

-0.0447 

1.1182 

EL 

4  65 

0.1293 

0.1310 

0.0018 

0.9865 

FA 

556 

0.1060 

0.1195 

0.0136 

0.8865 

GM 

552 

0.0521 

0.0598 

0.0077 

0.8705 

MM 

722 

0.2003 

0.2283 

0.0279 

0.8777 

OF 

795 

0.1211 

0.1356 

0.0145 

0.8929 

SC 

180 

0.1096 

0.0777 

-0.0319 

1 .4104 

ST 

619 

0.1774 

0.1748 

-0.0026 

1.0146 

Non-Acc 

1281 

0.1281 

0.1279 

-0.0002 

1.0016 

Cat4 

CL 

5 

0.6333 

0.7044 

0.0711 

0.8991 

CO 

2 

0.6361 

1.0000 

0.3639 

0.6361 

EL 

0 

. 

, 

, 

, 

FA 

1 

0.0725 

0.0000 

-0.0725 

, 

GM 

1 

0.1852 

0.0000 

-0.1852 

, 

MM 

2 

0.1425 

0.2086 

0.0661 

0.6830 

OF 

1 

0.3143 

0.0000 

-0.3143 

, 

SC 

1 

0.1244 

0.0000 

-0.1244 

, 

ST 

1 

0.0716 

0.0000 

-0.0716 

, 

Non-Acc 

6 

0.0994 

0.0000 

-0.0994 

. 
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Table  D.l.  Out-of-Sample  Fit  Diagnostics  for  First  Quarter  Model 


Subgroup 

Job 

Freq 

Estimate 

Actual 

Diff 

Ratio 

Family 

(E) 

(A) 

(A-E) 

(E/A) 

ALL 

CL 

6609 

0.1606 

0.1595 

-0.0011 

1.0066 

CO 

6448 

0.4323 

0.4050 

-0.0273 

1.0673 

EL 

5082 

0.1366 

0.1477 

0.0110 

0.9253 

FA 

5396 

0.1161 

0.1204 

0.0043 

0.9642 

CM 

4102 

0.0574 

0.0509 

-0.0066 

1.1293 

MM 

6346 

0.1485 

0.1570 

0.0084 

0.9462 

OF 

6078 

0.0798 

0.0825 

0.0027 

0.9670 

SC 

282 

0.3060 

0.2514 

-0.0546 

1.2172 

ST 

6318 

0.2618 

0.2552 

-0.0066 

1.0260 

Non-Acc 

10153 

0.1662 

0.1774 

0.0112 

0.9369 

Multiple-Opp 

CL 

6235 

0.1172 

0.1155 

-0.0017 

1.0147 

CO 

5272 

0.3399 

0.3066 

-0.0333 

1.1086 

EL 

4969 

0.1190 

0.1297 

0.0107 

0.9177 

FA 

5250 

0.0939 

0.0974 

0.0035 

0.9644 

GM 

4003 

0.0365 

0.0305 

-0.0060 

1.1976 

MM 

6197 

0.1275 

0.1345 

0.0071 

0.9476 

OF 

5981 

0.0661 

0.0691 

0.0030 

0.9566 

SC 

264 

0.2460 

0.1973 

-0.0487 

1.2470 

ST 

5755 

0.2035 

0.1957 

-0.0078 

1.0396 

Non-Acc 

7418 

0.1701 

0.1881 

0.0180 

0.9044 

Single-Opp 

CL 

374 

0.8195 

0.8284 

0.0089 

0.9893 

CO 

1176 

0.8681 

0.8693 

0.0012 

0.9986 

EL 

113 

0.8602 

0.8864 

0.0262 

0.9705 

FA 

146 

0.8577 

0.8900 

0.0324 

0.9636 

GM 

99 

0.7990 

0.7728 

-0.0262 

1.0338 

MM 

149 

0.8196 

0.8722 

0.0526 

0.9397 

OF 

97 

0.8185 

0.8063 

-0.0122 

1.0151 

SC 

18 

0.8613 

0.7523 

-0.1091 

1.1450 

ST 

563 

0.8298 

0.8341 

0.0043 

0.9948 

Non-Acc 

2735 

0.1559 

0.1493 

-0.0066 

1.0440 

Male 

CL 

5295 

0.1242 

0.1187 

-0.0055 

1.0466 

CO 

6060 

0.4577 

0.4291 

-0.0286 

1 .0667 

EL 

4169 

0.1405 

0.1543 

0.0138 

0.9108 

FA 

5211 

0.1197 

0.1245 

0.0048 

0.9612 

GM 

3056 

0.0588 

0.0503 

-0.0085 

1 . 1698 

MM 

5226 

0.1562 

0.1694 

0.0132 

0.9222 

OF 

5007 

0.0719 

0.0769 

0.0051 

0.9340 

SC 

216 

0.3075 

0.2898 

-0.0177 

1.0609 

ST 

4958 

0.2050 

0.2066 

0.0017 

0.9920 

Non-Acc 

8272 

0.1550 

0.1603 

0.0053 

0.9669 

Female 

CL 

1314 

0.3070 

0.3240 

0.0170 

0.9476 

CO 

388 

0.0405 

0.0342 

-0.0064 

1 . 1869 

EL 

913 

0.1188 

0.1174 

-0.0014 

1.0120 

FA 

185 

0.0110 

0.0000 

-0.0110 

GM 

1046 

0.0535 

0.0527 

-0.0008 

1 . 0156 

MM 

1120 

0.1125 

0.0986 

-0.0139 

1 .1404 

OF 

1071 

0.1167 

0.1085 

-0.0082 

1 . 0757 

SC 

66 

0.3009 

0.1155 

-0.1854 

2 . 6058 

ST 

1360 

0.4709 

0.4337 

-0.0371 

1 . 0856 

Non-Acc 

1881 

0.2152 

0.2521 

0.0369 

0.8535 
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Table  D.l.  Out-of-Sample  Fit  Diagnostics  for  First  Quarter  Model  (continued) 


Subgroup 

Job 

Family 

Freq 

Estimate 

(E) 

Actual 

(A) 

Diff 

(A-E) 

Ratio 

(E/A) 

HSGC 

CL 

3714 

0.1654 

0.1715 

0.0062 

0.9639 

CO 

3327 

0.4189 

0.3932 

-0.0257 

1.0653 

EL 

2663 

0.1330 

0.1414 

0.0084 

0.9405 

FA 

2937 

0.1110 

0.1187 

0.0077 

0.9355 

GM 

2281 

0.0620 

0.0489 

-0.0131 

1.2676 

MM 

3537 

0.1463 

0.1555 

0.0092 

0.9407 

OF 

3385 

0.0724 

0.0908 

0.0184 

0.7974 

SC 

80 

0.3674 

0.3696 

0.0022 

0.9940 

ST 

3339 

0.2574 

0.2443 

-0.0131 

1.0534 

Non-Acc 

5175 

0.1495 

0.1485 

-0.0010 

1.0065 

HSG 

CL 

774 

0.1409 

0.1598 

0.0188 

0.8823 

CO 

748 

0.3562 

0.3638 

0.0076 

0.9792 

EL 

642 

0.1326 

0.1531 

0.0205 

0.8663 

FA 

590 

0.0643 

0.0740 

0.0098 

0.8680 

GM 

523 

0.0436 

0.0511 

0.0075 

0.8536 

MM 

755 

0.1070 

0.0967 

-0.0103 

1.1060 

OF 

739 

0.0777 

0.0599 

-0.0177 

1.2960 

SC 

35 

0.3930 

0.4226 

0.0296 

0.9299 

ST 

857 

0.3360 

0.3673 

0.0313 

0.9148 

Non-Acc 

1224 

0.2272 

0.1867 

-0.0405 

1.2170 

Senior 

CL 

1307 

0.1702 

0.1256 

-0.0446 

1.3547 

CO 

1265 

0.3945 

0.3446 

-0.0499 

1.1448 

EL 

1090 

0.1087 

0.1165 

0.0078 

0.9328 

FA 

1123 

0.0660 

0.0700 

0.0040 

0.9430 

GM 

750 

0.0628 

0.0489 

-0.0139 

1.2843 

MM 

1205 

0.1167 

0.1414 

0.0246 

0.8258 

OF 

1189 

0.0595 

0.0462 

-0.0133 

1.2890 

SC 

121 

0.3044 

0.2047 

-0.0997 

1.4873 

ST 

1342 

0.2637 

0.2504 

-0.0134 

1.0534 

Non-Acc 

1998 

0.2197 

0.2868 

0.0671 

0.7662 

NG 

CL 

814 

0.1431 

0.1578 

0.0147 

0.9067 

CO 

1108 

0.5690 

0.5386 

-0.0304 

1.0564 

EL 

687 

0.1972 

0.2144 

0.0172 

0.9199 

FA 

746 

0.2514 

0.2385 

-0.0129 

1.0541 

GM 

548 

0.0449 

0.0612 

0.0163 

0.7333 

MM 

849 

0.2375 

0.2370 

-0.0005 

1.0020 

OF 

765 

0.1445 

0.1224 

-0.0221 

1.1810 

SC 

46 

0.1172 

0.0000 

-0.1172 

ST 

780 

0.1951 

0.1848 

-0.0104 

1.0562 

Non-Acc 

1756 

0.1131 

0.1358 

0.0227 

0.8330 
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Table  D.l.  Out-of-Sample  Fit  Diagnostics  for  First  Quarter  Model  (continued) 


Subgroup 

Job 

Family 

Freq 

Estimate 

(E) 

Actual 

(A) 

Diff 

(A-E) 

Ratio 

(E/A) 

Catl-3A 

CL 

4860 

0.1230 

0.1195 

-0.0036 

1.0299 

CO 

5134 

0.4192 

0.3947 

-0.0246 

1.0622 

EL 

4516 

0.1376 

0.1522 

0.0146 

0.9043 

FA 

4286 

0.1093 

0.1070 

-0.0023 

1.0210 

GM 

0.0392 

0.0368 

1.0661 

MM 

0.1389 

0.1374 

-0.0015 

1.0112 

OF 

0.0688 

0.0665 

SC 

183 

0.3358 

0.3549 

0.9463 

ST 

5216 

0.2740 

0.2644 

-0.0095 

Non-Acc 

7603 

0.1593 

0.1799 

0.0206 

0.8855 

Cat3B 

CL 

1743 

0.2674 

0.2736 

0.0063 

0.9771 

CO 

1307 

0.4838 

0.4471 

-0.0367 

1.0822 

EL 

564 

0.1290 

0.1116 

-0.0174 

1.1558 

FA 

1104 

0.1420 

0.1719 

0.0300 

0.8257 

GM 

733 

0.1410 

0.1157 

-0.0253 

1.2184 

MM 

1334 

0.1838 

0.2301 

0.0463 

0.7989 

OF 

1359 

0.1186 

0.1391 

0.0206 

0.8522 

SC 

99 

0.2462 

0.0437 

-0.2025 

5.6324 

ST 

1099 

0.2032 

0.2109 

0.0077 

0.9637 

Non-Acc 

2540 

0.1869 

0.1682 

-0.0187 

1 . 1112 

Cat4 

CL 

6 

0.0545 

0.0000 

-0.0545 

. 

CO 

7 

0.4397 

0.1571 

-0.2826 

2.7989 

EL 

2 

0.0539 

0.0000 

-0.0539 

FA 

6 

0.2074 

0.1815 

-0.0259 

1.1427 

GM 

4 

0.0736 

0.0000 

-0.0736 

, 

MM 

8 

0.3158 

0.2783 

-0.0375 

1.1349 

OF 

6 

0.0690 

0.0000 

-0.0690 

» 

SC 

. 

, 

, 

ST 

3 

0.0735 

0.0000 

-0.0735 

, 

Non-Acc 

10 

0.1882- 

0.5624 

0.3742 

0.3346 
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Table  D.2  Out-of-Sample  Fit  Diagnostics  for  Second  Quarter  Model 


Subgroup 

Job 

Family 

Freq 

Estimate 

(E) 

Actual 

(A) 

Diff 

(A-E) 

Ratio 

(E/A) 

ALL 

CL 

11682 

0.1985 

0.1954 

-0.0031 

1.0159 

CO 

11596 

0.4279 

0.3985 

-0.0295 

1.0739 

EL 

9382 

0.1405 

0.1539 

0.0134 

0.9131 

FA 

8652 

0.1261 

0.1307 

0.0046 

0.9648 

GM 

5427 

0.0426 

0.0420 

-0.0006 

1.0136 

MM 

10742 

0.1456 

0.1560 

0.0103 

0.9339 

OF 

10100 

0.0652 

0.0666 

0.0015 

0.9782 

SC 

7  67 

0.1749 

0.1892 

0.0143 

0.9242 

ST 

11309 

0.2543 

0.2480 

-0.0063 

1.0255 

Non-Acc 

17660 

0.1471 

0.1555 

0.0084 

0.9461 

Multiple-Opp 

CL 

10900 

0.1512 

0.1471 

-0.0041 

1.0276 

CO 

9539 

0.3318 

0.2945 

-0.0373 

1.1265 

EL 

9192 

0.1249 

0.1384 

0.0135 

0.9027 

FA 

8401 

0.1024 

0.1081 

0.0057 

0.9470 

GM 

5357 

0.0273 

0.0259 

-0.0014 

1.0556 

MM 

10491 

0.1269 

0.1365 

0.0096 

0.9300 

OF 

9927 

0.0504 

0.0519 

0.0014 

0.9724 

SC 

729 

0.1387 

0.1529 

0.0142 

0.9072 

ST 

10267 

0.1933 

0.1874 

-0.0060 

1.0319 

Non-Acc 

12806 

0.1502 

0.1636 

0.0134 

0.9184 

Single-Opp 

CL 

782 

0.8430 

0.8528 

0.0098 

0.9885 

CO 

2057 

0.8784 

0.8855 

0.0071 

0.9920 

EL  . 

190 

0.8439 

0.8528 

0.0089 

0.9896 

FA 

251 

0.8706 

0.8397 

-0.0309 

1.0368 

GM 

70 

0.8252 

0.8691 

0.0440 

0.9494 

MM 

251 

0.8504 

0.8888 

0.0384 

0.9568 

OF 

173 

0.8382 

0.8407 

0.0025 

0.9971 

SC 

38 

0.8443 

0.8615 

0.0171 

0.9801 

ST 

1042 

0.8527 

0.8429 

-0.0098 

1.0116 

Non-Acc 

4854 

0.1391 

0.1346 

-0.0044 

1.0329 

Male 

CL 

9077 

0.1464 

0.1420 

-0.0043 

1.0305 

CO 

10720 

0.4610 

0.4298 

-0.0312 

1.0726 

EL 

7546 

0.1391 

0.1542 

0.0151 

0.9022 

FA 

8464 

0.1280 

0.1328 

0.0049 

0.9635 

GM 

3831 

0.0502 

0.0508 

0.0006 

0.9886 

MM 

8676 

0.1597 

0.1661 

0.0064 

0.9612 

OF 

8106 

0.0562 

0.0569 

0.0007 

0.9872 

SC 

441 

0.1751 

0.2246 

0.0495 

0.7798 

ST 

8840 

0.2065 

0.2078 

0.0013 

0.9939 

Non-Acc 

14097 

0.1303 

0.1387 

0.0084 

0.9392 

Female 

CL 

2605 

0.3810 

0.3822 

0.0012 

0.9969 

CO 

876 

0.0243 

0.0161 

-0.0082 

1.5083 

EL 

1836 

0.1465 

0.1528 

0.0063 

0.9586 

FA 

188 

0.0395 

0.0326 

-0.0070 

1.2135 

GM 

1596 

0.0240 

0.0206 

-0.0034 

1.1631 

MM 

2066 

0.1129 

0.0267 

0.7638 

OF 

1994 

0.1066 

0.0044 

0.9585 

SC 

326 

0.1745 

0.1415 

-0.0330 

1.2333 

ST 

2469 

0.3931 

-0.0337 

1.0858 

Non-Acc 

3563 

0.2140 

0.2222 

0.0082 

0.9633 
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Table  D.2  Out-of-Sam  pie  Fit  Diagnostics  for  Second  Quarter  Model  (continued) 


Subgroup 

Job 

Family 

Freq 

Estimate 

(E) 

Actual 

(A) 

Diff 

(A-E) 

Ratio 

(E/A) 

HSGC 

CL 

6477 

0.2044 

0.2153 

0.0109 

0.9493 

CO 

5960 

0.4052 

0.3780 

-0.0272 

1.0721 

EL 

4875 

0.1239 

0.1415 

0.0177 

0.8753 

FA 

4695 

0.1319 

0.1257 

-0.0061 

1.0487 

GM 

3037 

0.0436 

0.0449 

0.0013 

0.9721 

MM 

5782 

0.1387 

0.1501 

0.0114 

0.9240 

OF 

5340 

0.0665 

0.0652 

-0.0013 

1.0192 

SC 

233 

0.2169 

0.2792 

0.0623 

0.7768 

ST 

6026 

0.2355 

0.2222 

-0.0134 

1.0601 

Non-Acc 

8739 

0.1218 

0.1259 

0.0041 

0.9672 

HSG 

CL 

1444 

0.2015 

0.1828 

-0.0188 

1.1027 

CO 

1313 

0.3668 

0.3002 

-0.0666 

1.2217 

EL 

1217 

0.1524 

0.1593 

0.0069 

0.9568 

FA 

1024 

0.0673 

0.0879 

0.0206 

0.7658 

GM 

730 

0.0298 

0.0351 

0.0052 

0.8510 

MM 

1335 

0.1269 

0.0999 

-0,0270 

1.2699 

OF 

1268 

0.0642 

0.0576 

-0.0066 

1.1146 

SC 

71 

0.2541 

0.2734 

0.0193 

0.9293 

ST 

1603 

0.3447 

0.3540 

0.0093 

0.9738 

Non-Acc 

2171 

0.1457 

0.1955 

0.0498 

0.7454 

Senior 

CL 

2498 

0.2098 

0.1688 

-0.0411 

1.2434 

CO 

2455 

0.3896 

0.3530 

-0.0366 

1.1037 

EL 

2113 

0.1421 

0.1348 

-0.0073 

1.0539 

FA 

1770 

0.0588 

0.0879 

0.0291 

0.6692 

GM 

1028 

0.0516 

0.0395 

-0.0121 

1 . 3061 

MM 

2325 

0.1206 

0.1479 

0.0273 

0.8157 

OF 

2311 

0.0486 

0.0508 

0.0022 

0.9565 

SC 

377 

0.1325 

0.1459 

0.0134 

0.9081 

ST 

2437 

0.2327 

0.2584 

0.0257 

0.9004 

Non-Acc 

3773 

0.2214 

0.2300 

0.0086 

0.9628 

NG 

CL 

1263 

0.1435 

0.1603 

0.0169 

0.8947 

CO 

1868 

0.5940 

0.5929 

-0.0011 

1 . 0018 

EL 

1177 

0.1945 

0.2331 

0.0387 

0.8342 

FA 

1163 

0.2547 

0.2519 

-0.0028 

1 . 0111 

GM 

632 

0.0378 

0.0403 

0.0025 

0.9384 

MM 

1300 

0.2390 

0.2525 

0.0135 

0.9464 

OF 

1181 

0.0922 

0.1129 

0.0207 

0.8163 

SC 

86 

0.1780 

0.0623 

-0.1157 

2.8584 

ST 

1243 

0.2703 

0.2161 

-0.0542 

1.2508 

Non-Acc 

2977 

0.1298 

0.1204 

-0.0095 

1.0788 
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Table  D.2  Out-of-Sample  Fit  Diagnostics  for  Second  Quarter  Model  (continued) 


Subgroup 

Job 

Family 

Freq 

Estimate 

(E) 

Actual 

(A) 

Diff 

(A-E) 

Ratio 

(E/A) 

Catl-3A 

CL 

7980 

0.1453 

0.1345 

-0.0108 

1.0804 

CO 

8591 

0.3985 

0.3749 

-0.0236 

1.0629 

EL 

7806 

0.1462 

0.1630 

0.0168 

0.8972 

FA 

6753 

0.1163 

0.1220 

0.0058 

0.9528 

GM 

4208 

0.0354 

0.0349 

-0.0005 

1.0146 

MM 

8219 

0.1367 

0.1409 

0.0042 

0.9699 

OF 

7736 

0.0538 

0.0546 

0.0007 

0.9870 

SC 

530 

0.1857 

0.2309 

0.0452 

0.8044 

ST 

8947 

0.2640 

0.2612 

-0.0027 

1.0105 

Non-Acc 

12511 

0.1544 

0.1608 

0.0064 

0.9603 

Cat3B 

CL 

3571 

0.3121 

0.3238 

0.0117 

0.9638 

CO 

2868 

0.5052 

0.4528 

-0.0524 

1.1156 

EL 

1548 

0.1133 

0.1107 

-0.0026 

1.0234 

FA 

1857 

0.1581 

0.1575 

-0.0006 

1.0038 

GM 

1164 

0.0645 

0.0692 

0.0047 

0.9322 

MM 

2448 

0.1737 

0.2041 

0.0305 

0.8509 

OF 

2272 

0.0996 

0.1080 

0.0085 

0.9217 

SC 

233 

0.1525 

0.0976 

-0.0549 

1.5628 

ST 

2337 

0.2183 

0.1989 

-0.0194 

1.0975 

Non-Acc 

4  927 

0.1300 

0.1445 

0.0145 

0.8997 

Cat4 

CL 

131 

0.3780 

0.4439 

0.0659 

0.8515 

CO 

137 

0.6676 

0.7484 

0.0808 

0.8920 

EL 

28 

0.0549 

0.0000 

-0.0549 

, 

FA 

42 

0.2875 

0.3361 

0.0486 

0.8554 

GM 

55 

0.1366 

0.0192 

-0.1174 

7.1342 

MM 

75 

0.2135 

0.2343 

0.0207 

0.9115 

OF 

92 

0.1772 

0.0683 

-0.1089 

2.5942 

SC 

4 

0.0342 

0.0000 

-0.0342 

ST 

25 

0.1225 

o 

o 

o 

-0.0822 

3.0348 

Non-Acc 

222 

0.1130 

0.0992 

-0.0139 

1.1401 
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Table  D.3.  Out-of-Sample  Fit  Diagnostics  for  Third  Quarter  Model 


Subgroup 

Job 

Freq 

Estimate 

Actual 

Diff 

Ratio 

Family 

(E) 

(A) 

(A-E) 

(E/A) 

ALL 

CL 

13751 

0.2232 

0.2179 

-0.0053 

1.0245 

CO 

10001 

0.3935 

0.3637 

-0.0297 

1.0818 

EL 

9611 

0.1493 

0.1632 

0.0139 

0.9147 

FA 

9837 

0.1353 

0.1456 

0.0102 

0.9297 

GM 

6739 

0.0404 

0.0359 

-0.0045 

1.1240 

MM 

12651 

0.1826 

0.1836 

0.0010 

0.9946 

OF 

12591 

0.1072 

0.1073 

0.0001 

0.9991 

SC 

533 

0.1898 

0.1802 

-0.0096 

1.0534 

ST 

11750 

0.2664 

0.2477 

-0.0188 

1 . 0757 

Non-Acc 

19868 

0.1507 

0.1697 

0.0191 

0.8877 

Multiple-Opp 

CL 

12827 

0.1771 

0.1723 

-0.0048 

1.0277 

CO 

8587 

0.3071 

0.2738 

-0.0334 

1 . 1218 

EL 

9445 

0.1362 

0.1506 

0.0144 

0.9042 

FA 

9641 

0.1208 

0.1309 

0.0102 

0.9223 

GM 

6677 

0.0289 

0.0237 

-0.0053 

1.2232 

MM 

12325 

0.1646 

0.1660 

0.0014 

0.9914 

OF 

12242 

0.0862 

0.0863 

0.0001 

0.9990 

SC 

511 

0.1544 

0.1468 

-0.0076 

1.0520 

ST 

10729 

0.2075 

0.1909 

-0.0166 

1 . 0871 

Non-Acc 

15388 

0.1568 

0.1772 

0.0204 

0.8851 

Single-Opp 

CL 

924 

0.8531 

0.8401 

-0.0130 

1.0154 

CO 

1414 

0.8935 

0.8847 

-0.0088 

1.0100 

EL 

166 

0.8572 

0.8442 

-0.0130 

1.0154 

FA 

196 

0.8680 

0.8811 

0.0131 

0.9851 

GM 

62 

0.8658 

0.9211 

0.0553 

0.9400 

MM 

326 

0.8486 

0.8337 

-0.0149 

1.0179 

OF 

349 

0.8449 

0.8451 

0.0002 

0.9998 

SC 

22 

0.8780 

0.8298 

-0.0482 

1 . 0581 

ST 

1021 

0.8682 

0.8278 

-0.0404 

1 . 0489 

Non-Acc 

4480 

0.1303 

0.1451 

0.0148 

0.8982 

Male 

CL 

10812 

0.1794 

0.1715 

-0.0078 

1.0457 

CO 

9780 

0.4014 

0.3709 

-0.0305 

1 . 0823 

EL 

7308 

0.1458 

0.1608 

0.0150 

0.9068 

FA 

9697 

0.1370 

0.1477 

0.0107 

0.9279 

GM 

4814 

0.0488 

0.0425 

-0.0063 

1.1480 

MM 

10144 

0.1933 

0.1961 

0.0028 

0.9857 

OF 

10224 

0.0985 

0.1015 

0.0031 

0.9699 

SC 

371 

0.1343 

0.1486 

0.0143 

0.9039 

ST 

9131 

0.2268 

0.2078 

-0.0190 

1.0914 

Non-Acc 

15621 

0.1337 

0.1536 

0.0198 

0.8709 

Female 

CL 

2939 

0.3850 

0.3889 

0.0039 

0.9900 

CO 

221 

0.0531 

0.0577 

0.0046 

0.9201 

EL 

2303 

0.1605 

0.1710 

0.0105 

0.9385 

FA 

140 

0.0184 

0.0000 

-0.0184 

GM 

1925 

0.0189 

0.0191 

0.0002 

0.9886 

MM 

2507 

0.1392 

0.1328 

-0.0064 

1.0482 

OF 

2367 

0.1453 

0.1325 

-0.0128 

1 . 0968 

SC 

162 

0.3150 

0.2515 

-0.0635 

1.2526 

ST 

2619 

0.4048 

0.3868 

-0.0179 

1.0463 

Non-Acc 

4247 

0;2132 

0.2295 

0.0163 

0. 929l| 
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Table  D.3.  Out-of-SampIe  Fit  Diagnostics  for  Third  Quarter  Model  (continued) 


Subgroup 

Job 

Family 

Freq 

Estimate 

(E) 

Actual 

(A) 

Diff 

(A-E) 

Ratio 

(E/A) 

HSGC 

CL 

6831 

0.2461 

0.2426 

-0.0034 

1.0142 

CO 

4567 

0.3587 

0.3388 

-0.0199 

1.0587 

EL 

4419 

0.1478 

0.1598 

0.0120 

0.9250 

FA 

4704 

0.1291 

0.1450 

0.0159 

0.8901 

GM 

3331 

0.0415 

0.0407 

-0.0008 

1.0192 

MM 

6030 

0.1888 

0.1858 

-0.0029 

1.0158 

OF 

6168 

0.1117 

0.1124 

0.0007 

0.9940 

SC 

287 

0.1798 

0.1539 

-0.0259 

1.1680 

ST 

5360 

0.2746 

0.2477 

-0.0269 

1.1087 

Non-Acc 

9296 

0.1350 

0.1516 

0.0166 

0.8904 

HSG 

CL 

1689 

0.2417 

0.2246 

-0.0171 

1.0763 

CO 

1172 

0.3857 

0.3446 

-0.0412 

1.1195 

EL 

1251 

0.1652 

0.1814 

0.0162 

0.9108 

FA 

1130 

0.1035 

0.0920 

-0.0115 

1.1252 

GM 

914 

0.0246 

0.0195 

-0.0051 

1.2594 

MM 

1567 

0.0923 

0.1027 

0.0104 

0.8989 

OF 

1523 

0.0857 

0.0567 

-0.0290 

1.5113 

SC 

60 

0.2632 

0.2736 

0.0104 

0.9619 

ST 

1703 

0.3767 

0.3496 

-0.0271 

1.0776 

Non-Acc 

2480 

0.1457 

0.2049 

0.0591 

0.7113 

Senior 

CL 

2941 

0.2069 

0.1911 

-0.0158 

1.0827 

CO 

2574 

0.4330 

0.3800 

-0.0530 

1.1396 

EL 

2319 

0.1443 

0.1725 

0.0282 

0.8365 

FA 

2042 

0.0745 

0.0788 

0.0043 

0.9458 

GM 

1403 

0.0407 

0.0282 

-0.0125 

1.4437 

MM 

2680 

0.1468 

0.1565 

0.0097 

0.9379 

OF 

2625 

0.0819 

0.0730 

-0.0089 

1.1215 

SC 

68 

0.2659 

0.3984 

0.1325 

0.6675 

ST 

2673 

0.2425 

0.2316 

-0.0109 

1.0469 

Non-Acc 

4334 

0.1848 

0.2179 

0.0331 

0.8482 

NG 

CL 

2290 

0.1622 

0.1733 

0.0111 

0.9361 

CO 

1688 

0.4330 

0.4196 

-0.0134 

1.0319 

EL 

1622 

0.1483 

0.1455 

-0.0028 

1.0194 

FA 

1961 

0.2315 

0.2467 

0.0152 

0.9385 

GM 

1091 

0.0494 

0.0445 

-0.0049 

1.1106 

MM 

2374 

0.2660 

0.2611 

-0.0049 

1.0188 

OF 

2275 

0.1383 

0.1663 

0.0280 

0.8316 

SC 

118 

0.1290 

0.0629 

-0.0661 

2.0507 

ST 

2014 

0.1828 

0.1825 

-0.0003 

1.0014 

Non-Acc 

3758 

0.1536 

0.1362 

-0.0175 

1.1281 
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Table  D.3.  Out-of-Sample  Fit  Diagnostics  for  Third  Quarter  Model  (continued) 


Subgroup 

Job 

Family 

Freq 

Estimate 

(E) 

Actual 

(A) 

Diff 

(A-E) 

Ratio 

(E/A) 

Catl-3A 

CL 

9138 

0.1648 

0.1592 

-0.0057 

1.0357 

CO 

6839 

0.3757 

0.3369 

-0.0388 

1.1151 

EL 

7858 

0.1515 

0.1675 

0.0160 

0.9047 

FA 

7438 

0.1175 

0.1295 

0.0120 

0.9073 

GM 

5349 

0.0326 

0.0283 

-0.0043 

1.1517 

MM 

9569 

0.1516 

0.1599 

0.0083 

0.9482 

OF 

9093 

0.0862 

0.0819 

-0.0043 

1.0521 

SC 

255 

0.2779 

0.3126 

0.0347 

0.8889 

ST 

9416 

0.2833 

0.2580 

-0.0254 

1.0983 

Non-Acc 

13220 

0.1508 

0.1746 

0.0238 

0.8638 

Cat3B 

CL 

4292 

0.3394 

0.3335 

-0.0059 

1.0176 

CO 

2849 

0.4165 

0.3972 

-0.0193 

1 . 0485 

EL 

1672 

0.1418 

0.1493 

0.0075 

0.9500 

FA 

2307 

0.1917 

0.1937 

0.0021 

0.9893 

GM 

1225 

0.0658 

0.0671 

0.0013 

0.9800 

MM 

2855 

0.2719 

0.2546 

-0.0174 

1 .0681 

OF 

3196 

0.1563 

0.1691 

0.0129 

0.9238 

SC 

255 

0.1117 

0.0637 

-0.0481 

1.7551 

ST 

2267 

0.1970 

0.2047 

0.0078 

0.9621 

Non-Acc 

6073 

0.1524 

0.1632 

0.0108 

0.9341 

Cat4 

CL 

321 

0.3475 

0.3595 

0.0119 

0.9669 

CO 

313 

0.5695 

0.6404 

0.0709 

0.8893 

EL 

81 

0.0874 

0.0358 

-0 . 0516 

•  2.4416 

FA 

92 

0.1749 

0.2462 

0.0713 

0.7104 

GM 

165 

0.1044 

0.0520 

-0.0524 

2 . 0069 

MM 

227 

0.3780 

0.3008 

-0.0772 

1.2566 

OF 

302 

0.2292 

0.2266 

-0.0026 

1.0114 

SC 

23 

0.0758 

0.0000 

-0.0758 

, 

ST 

67 

0.2152 

0.2377 

0.0225 

0.9053 

Non-Acc 

575 

0.1295 

0.1271 

-0.0024 

1.0186 
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Table  D.4.  Out-of-Sample  Fit  Diagnostics  for  Fourth  Quarter  Model 


Subgroup 

Job 

Family 

Freq 

Estimate 

(E) 

Actual 

(A) 

Diff 

(A-E) 

Ratio 

(E/A) 

ALL 

CL  , 

18222 

0.2081 

0.2101 

0.0020 

0.9906 

j 

CO 

17542 

0.3867 

0.3525 

-0.0342 

1.0970 

EL 

14903 

0.1280 

0.1343 

0.0064 

0.9524 

! 

FA 

14410 

0.0996 

0.0995 

-0.0001 

1.0006 

GM 

13326 

0.0351 

0.0360 

0.0009 

0.9757 

MM 

17898 

0.1535 

0.1619 

0.0084 

0.9481 

OF 

17408 

0.0862 

0.0849 

-0.0013 

1.0156 

SC 

2799 

0.1904 

0.1884 

-0.0019 

1.0103 

ST 

19047 

0.2857 

0.2717 

-0.0140 

1.0515 

Non-Acc 

27982 

0.1197 

0.1413 

0.0215 

0.8475 

Multiple-Opp 

CL 

17298 

0.1720 

0.1750 

0.0030 

0.9828 

CO 

15506 

0.3139 

0.2802 

-0.0337 

1.1204 

EL 

14629 

0.1142 

0.1207 

0.0065 

0.9460 

FA 

14243 

0.0903 

0.0900 

-0.0003 

1.0034 

GM 

13217 

0.0283 

0.0286 

0.0004 

0.9878 

MM 

17551 

0.1399 

0.1482 

0.0083 

0.9441 

OF 

17058 

0.0709 

0.0693 

-0.0015 

1.0222 

SC 

2719 

0.1684 

0.1676 

-0.0008 

1.0048 

ST 

17579 

0.2346 

0.2230 

-0.0116 

1.0519 

Non-Acc 

22227 

0.1254 

0.1461 

0.0208 

0.8578 

Single-Opp 

CL 

924 

0.8863 

0.8686 

-0.0177 

1.0203 

CO 

2036 

0.9379 

0.9003 

-0.0376 

1.0418 

EL 

274 

0.8684 

0.8681 

-0.0003 

1.0004 

FA 

167 

0.8604 

0.8806 

0.0202 

0.9771 

GM 

109 

0.7791 

0.8371 

0.0580 

0.9307 

MM 

347 

0.8330 

0.8466 

0.0136 

0.9840 

OF 

350 

0.8387 

0.8477 

0.0090 

0.9893 

SC 

80 

0.9449 

0.9043 

-0.0406 

1.0449 

ST 

1468 

0.9129 

0.8694 

-0.0436 

1.0501 

Non-Acc 

5755 

0.0979 

0.1224 

0.0245 

0.8002 

Male 

CL 

14304 

0.1495 

0.1425 

-0.0070 

1.0490 

CO 

16245 

0.4152 

0.3784 

-0.0368 

1.0971 

EL 

11863 

0.1304 

0.1382 

0.0078 

0.9434 

FA 

13910 

0.1027 

0.1025 

-0.0002 

1.0022 

GM 

10088 

0.0338 

0.0341 

0.0004 

0.9895 

MM 

14500 

0.1644 

0.1754 

0.0110 

0.9373 

OF 

13939 

0.0693 

0.0716 

0.0022 

0.9687 

SC 

1668 

0.1703 

0.2094 

0.0391 

0.8131 

ST 

14647 

0.2342 

0.2190 

-0.0153 

1.0697 

Non-Acc 

21675 

0.1106 

0.1370 

0.0263 

0.8077 

Table  D.4.  Out-of-Sample  Fit  Diagnostics  for  Fourth  Quarter  Model(contiuued) 


Subgroup 

Job 

Family 

Freq 

Estimate 

(E) 

Actual 

(A) 

Diff 

(A-E) 

Ratio 

(E/A) 

HSGC 

CL 

9720 

0.2262 

0.2324 

0.0062 

0.9733 

CO 

9158 

0.3672 

0.3146 

-0.0526 

1.1670 

EL 

7338 

0.1107 

0.1298 

0.0191 

0.8532 

FA 

7912 

0.1068 

0.1079 

0.0011 

0.9896 

GM 

7507 

0.0378 

0.0403 

0.0024 

0.9394 

MM 

9418 

0.1569 

0.1674 

0.0105 

0.9372 

OF 

9194 

0.0921 

0.0930 

0.0009 

0.9904 

SC 

1356 

0.2012 

0.2033 

0.0021 

0.9897 

ST 

9996 

0.2886 

0.2576 

-0.0311 

1 . 1205 

Non-Acc 

14696 

0.1156 

0.1465 

0.0309 

0.7894 

HSG 

CL 

1933 

0.2390 

0.2184 

-0.0207 

1.0947 

CO 

1771 

0.3592 

0.3005 

-0.0587 

1.1952 

EL 

1637 

0.1037 

0.1192 

0.0154 

0.8705 

FA 

1516 

0.0452 

0.0671 

0.0219 

0.6736 

GM 

1503 

0.0151 

0.0215 

0.0064 

0.7013 

MM 

1923 

0.0872 

0.0994 

0.0122 

0.8769 

OF 

1780 

0.0669 

0.0591 

-0.0079 

1 .1337 

SC 

253 

0.2772 

0.2250 

-0.0522 

1.2319 

ST 

0.3892 

0.3856 

-0.0036 

1 . 0093 

Non-Acc 

3119 

0.1504 

0.1787 

0.0283 

0.8417 

Senior 

CL 

5254 

0.1808 

0.1805 

-0.0003 

1 . 0015 

CO 

5060 

0.4205 

0.4150 

-0.0055 

1.0132 

EL 

4626 

0.1401 

0.1338 

-0.0063 

1 . 0471 

FA 

3800 

0.0873 

0.0715 

-0.0158 

1.2207 

GM 

3330 

0.0426 

0.0338 

-0.0088 

1.2595 

MM 

5116 

0.1584 

0.1626 

0.0042 

0.9742 

OF 

5105 

0.0739 

0.0709 

-0.0030 

1.0426 

SC 

1028 

0.1603 

0.1799 

0.0196 

0.8912 

ST 

5225 

0.2743 

0.2732 

-0.0011 

1.0041 

Non-Acc 

7732 

0.0958 

0.1123 

0.0165 

0.8531 

NG 

CL 

1315 

0.1387 

0.1515 

0.0128 

0.9156 

CO 

1553 

0.4225 

0.4303 

0 . 0078 

0.9818 

EL 

1302 

0.2120 

0.1809 

-0.0312 

1 . 1723 

FA 

1182 

0.1599 

0.1745 

0.0146 

0.9164 

GM 

986 

0.0198 

0.0329 

0.0131 

0.6010 

MM 

1441 

0.2018 

0.2061 

0.0044 

0.9788 

OF 

1329 

0.1182 

0.1169 

-0.0014 

1.0116 

SC 

162 

0.1553 

0.0614 

-0.0939 

2.5304 

ST 

1426 

0.1344 

0.1750 

0.0406 

0.7679 

Non-Acc 

2435 

0.1813 

0.1541 

-0 . 0271 

1.1761 
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Table  D.4,  Out-of-Sample  Fit  Diagnostics  for  Fourth  Quarter  Model(continued) 


Subgroup 

Job 

Family 

Freq 

Estimate 

(E) 

Actual 

(A) 

Diff 

(A-E) 

Ratio 

(E/A) 

Catl-3A 

CL 

12897 

0.1500 

0.1460 

-0.0040 

1.0273 

CO 

12511 

0.3553 

0.3380 

-0.0173 

1.0512 

EL 

12293 

0.1267 

0.1343 

0.0075 

0.9440 

FA 

11190 

0.0954 

0.0940 

-0.0014 

1.0153 

GM 

10193 

0.0289 

0.0277 

-0.0012 

1.0425 

MM 

13855 

0.1357 

0.1335 

-0.0023 

1.0168 

OF 

12753 

0.0683 

0.0616 

-0.0067 

1.1084 

SC 

1821 

0.2265 

0.2315 

0.0051 

0.9782 

ST 

15353 

0.3055 

0.2891 

-0.0164 

1.0568 

Non-Acc 

19476 

0.1188 

0.1477 

0.0289 

0.8043 

Cat3B 

CL 

5305 

0.3476 

0.3635 

0.0159 

0.9562 

CO 

5013 

0.4634 

0.3877 

-0.0758 

1.1955 

EL 

2607 

0.1337 

0.1349 

0.0012 

0.9909 

FA 

3214 

0.1136 

0.1184 

0.0048 

0.9595 

GM 

3128 

0.0553 

0.0629 

0.0076 

0.8798 

MM 

4034 

0.2134 

0.2583 

0.0449 

0.8263 

OF 

4640 

0.1346 

0.1478 

0.0132 

0.9109 

SC 

975 

0.1237 

0.1091 

-0.0146 

1.1338 

ST 

3687 

0.2041 

0.2002 

-0.0039 

1.0194 

Non-Acc 

8467 

0.1220 

0.1266 

0.0046 

0.9637 

Cat4 

CL 

20 

0.5710 

0.7006 

0.1296 

0.8150 

CO 

18 

0.7127 

0.5720 

-0.1407 

1.2459 

EL 

3 

0.0840 

0.0000 

-0.0840 

FA 

6 

0.2327 

0.1691 

-0.0637 

1.3765 

GM 

5 

0.0330 

0.0000 

-0.0330 

, 

MM 

9 

0.3412 

0.2230 

-0.1182 

1.5303 

OF 

15 

0.2815 

0.3363 

0.0548 

0.8370 

SC 

3 

0.1057 

0.0000 

-0.1057 

, 

ST 

7 

0.1737 

0.1422 

-0.0316 

1.2220 

Non-Acc 

39 

0.1066 

0.1467 

0.0401 

0.7268 
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